Target sorting layers as assets [Repost]

  • Post comments:0 Comments

[This was originally posted in the Unity forums]

I come from here: https://forum.unity.com/threads/2d-lights-proposal-target-sorting-layers-as-an-asset.1024630/

The problem, summarizing

Target sorting layers is a property used by Light2Ds and ShadowCaster2Ds, which determine which sorting layers will be affected by them. The content of this property may be the same for hundreds of lights/shadow casters in a videogame project, some of them stored in prefabs and some of them stored in scenes. Real case: Imagine you have 3 sorting layers, Walls, Characters and Background. You also have 60 scenes, 200 lights in total. 150 of them have Characters and Background layers enabled and Walls disabled. Then you think that it would be good to add another sorting layer, Items, that should be affected by all those 150 lights. GOOD LUCK.

“But you could use a prefab for lights to avoid having to change them one by one!” NOPE. Imagine you have many different lights of different types and with different settings, having 50 prefabs is not practical.
Besides, what happens if there are light instances in the scene that override the Target sorting layers property of the prefab? You cannot affect that value anymore by changing the prefab.
It would not be rare, also, that the Target sorting layers in some shadow casters are necessarily the same as the lights’.

The solution, so far

If you could store just a few predefined Target sorting layers configurations, and reference them from every light prefab or instance, then you would only need to change those assets to affect all the lights in your project.

That’s what I’ve implemented and share with you in this post. Please feel free to propose improvements or report bugs. Beware of Unity changing how they call the internal variables of their classes, my code makes some assumptions.

How it works

  1. Create a TargetSortingLayers asset.
  2. Enable / disable the desired sorting layers.
  3. Add a TargetSortingLayersSetter component to every light and every shadow caster in your project (I recommend to create a prefab that already includes the component).
  4. Fill the property that references the asset.
  5. When the scene is loaded, in the Awake function of the light, the Target sorting layers will replaced.

    // Copyright 2021 Alejandro Villalba Avila
    //
    // Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"),
    // to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
    // and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
    //
    // The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
    //
    // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
    // IN THE SOFTWARE.
     
    #if ODIN_INSPECTOR
    using Sirenix.OdinInspector;
    #endif
    using UnityEngine;
    using UnityEngine.Experimental.Rendering.Universal;
     
    namespace Game.Utils
    {
        /// <summary>
        /// A component that just replaced the Target sorting layers with those stored in an asset as soon as the game object awakes.
        /// </summary>
        /// <remarks>
        /// By using this component in every light or shadow caster, you only need to define a set of TargetSortingLayers assets and reference them.
        /// If you need to add / remove sorting layers, or change whether they are enabled / disabled for a group of lights, you just need to change the asset
        /// and all those lights will be affected when the scene is loaded. Otherwise, you would have to visit light by light, in every scene, and change its
        /// Target sorting layers property by hand.
        /// It's highly recommended that you create a prefab per type of light and shadow caster, which have this component already added along with a default asset.
        /// </remarks>
        public class TargetSortingLayersSetter : MonoBehaviour
        {
            [Tooltip("The asset that contains the sorting layers configuration that will replace the one of the target component, when awaking.")]
            [SerializeField]
            protected TargetSortingLayers m_TargetSortingLayers;
     
            [Tooltip("The target component whose uses a property to store the Target sorting layers. It may be a Light2D or a ShadowCaster2D, for example.")]
            [SerializeField]
            protected MonoBehaviour m_TargetComponent;
     
            private void Reset()
            {
                // Automatically gets the component that uses Target sorting layers property, although it can be set manually afterwards
                Light2D lightComponent;
                ShadowCaster2D shadowCasterComponent;
     
                if (TryGetComponent(out lightComponent))
                {
                    m_TargetComponent = lightComponent;
                }
                else if(TryGetComponent(out shadowCasterComponent))
                {
                    m_TargetComponent = shadowCasterComponent;
                }
                else
                {
                    Debug.LogError("There is no component in the game object '" + name + "' that uses Target Sorting Layers.");
                }
            }
     
            private void Awake()
            {
                SetTargetSortingLayers();
            }
     
            /// <summary>
            /// Replaces the content of the Target sorting layers property of the target component with the content of the asset.
            /// </summary>
    #if ODIN_INSPECTOR
            [GUIColor(0.0f, 1.0f, 0.0f)]
            [Button("Set Target sorting layers")]
    #endif
            public void SetTargetSortingLayers()
            {
                if(m_TargetSortingLayers == null)
                {
                    Debug.LogError("There is no TargetSortingLayers asset in the game object '" + name + "'.");
                }
                else
                {
                    if (m_TargetComponent is Light2D)
                    {
                        (m_TargetComponent as Light2D).SetTargetSortingLayers(m_TargetSortingLayers.SortingLayers.ToArray());
                    }
                    else if (m_TargetComponent is ShadowCaster2D)
                    {
                        (m_TargetComponent as ShadowCaster2D).SetTargetSortingLayers(m_TargetSortingLayers.SortingLayers.ToArray());
                    }
                    else
                    {
                        Debug.LogError("There is no component in the game object '" + name + "' that uses Target Sorting Layers.");
                    }
                }
            }
        }
    }
    // Copyright 2021 Alejandro Villalba Avila
    //
    // Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"),
    // to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
    // and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
    //
    // The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
    //
    // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
    // IN THE SOFTWARE.
     
    using UnityEngine;
    using System.Collections.Generic;
    using UnityEngine.UIElements;
     
    #if UNITY_EDITOR
    using UnityEditor;
    #endif
     
    namespace Game.Utils
    {
        /// <summary>
        /// Stores a list of enabled and disabled sorting layers.
        /// </summary>
        [CreateAssetMenu(fileName = "NewTargetSortingLayers", menuName = "Game/Utils/Target sorting layers")]
        public class TargetSortingLayers : ScriptableObject
        {
            /// <summary>
            /// A list of enabled sorting layer IDs.
            /// </summary>
            public List<int> SortingLayers = new List<int>();
     
            // Just for debugging and tracking code repository changes easier
            protected List<string> m_SortingLayerNames = new List<string>();
     
    #if UNITY_EDITOR
     
            [CustomEditor(typeof(TargetSortingLayers))]
            protected class CustomTargetSortingLayersInspector : Editor
            {
                private bool[] m_enabledSortingLayers;
                private SortingLayer[] m_editorSortingLayers;
                private TargetSortingLayers m_target;
     
                public override VisualElement CreateInspectorGUI()
                {
                    m_target = target as TargetSortingLayers;
                    m_editorSortingLayers = SortingLayer.layers;
                    m_enabledSortingLayers = new bool[m_editorSortingLayers.Length];
     
                    for (int i = 0; i < m_enabledSortingLayers.Length; ++i)
                    {
                        m_enabledSortingLayers[i] = m_target.SortingLayers.Contains(m_editorSortingLayers[i].id);
                    }
     
                    return null;
                }
             
                public override void OnInspectorGUI()
                {
                    bool hasSomethingChanged = false;
     
                    // Draws the list of sorting layer checkboxes
                    EditorGUILayout.BeginVertical();
                    {
                        for(int i = 0; i < m_editorSortingLayers.Length; ++i)
                        {
                            EditorGUILayout.BeginHorizontal();
                            {
                                EditorGUI.BeginChangeCheck();
                                {
                                    m_enabledSortingLayers[i] = EditorGUILayout.ToggleLeft(new GUIContent(m_editorSortingLayers[i].name), m_enabledSortingLayers[i]);
                                }
                                if (EditorGUI.EndChangeCheck())
                                {
                                    hasSomethingChanged = true;
                                }
                            }
                            EditorGUILayout.EndHorizontal();
                        }
                    }
                    EditorGUILayout.EndVertical();
     
                    // Draws some selection utility buttons
                    if(GUILayout.Button("All"))
                    {
                        for (int i = 0; i < m_editorSortingLayers.Length; ++i)
                        {
                            hasSomethingChanged = true;
                            m_enabledSortingLayers[i] = true;
                        }
                    }
     
                    if(GUILayout.Button("None"))
                    {
                        for (int i = 0; i < m_editorSortingLayers.Length; ++i)
                        {
                            hasSomethingChanged = true;
                            m_enabledSortingLayers[i] = false;
                        }
                    }
     
                    // Serializes if necessary
                    if(hasSomethingChanged)
                    {
                        m_target.SortingLayers.Clear();
                        m_target.m_SortingLayerNames.Clear();
                        SerializedProperty sortingLayersProp = serializedObject.FindProperty("SortingLayers");
                        SerializedProperty sortingLayerNamesProp = serializedObject.FindProperty("SortingLayerNames");
                        sortingLayersProp.ClearArray();
                        sortingLayerNamesProp.ClearArray();
     
                        for (int i = 0; i < m_enabledSortingLayers.Length; ++i)
                        {
                            if (m_enabledSortingLayers[i])
                            {
                                sortingLayersProp.InsertArrayElementAtIndex(0);
                                sortingLayersProp.GetArrayElementAtIndex(0).intValue = m_editorSortingLayers[i].id;
                                sortingLayerNamesProp.InsertArrayElementAtIndex(0);
                                sortingLayerNamesProp.GetArrayElementAtIndex(0).stringValue = m_editorSortingLayers[i].name;
     
                                m_target.SortingLayers.Add(m_editorSortingLayers[i].id);
                                m_target.m_SortingLayerNames.Add(m_editorSortingLayers[i].name);
                            }
                        }
     
                        serializedObject.ApplyModifiedProperties();
                    }
                }
            }
     
    #endif
     
        }
     
    }
    // Copyright 2021 Alejandro Villalba Avila
    //
    // Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"),
    // to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
    // and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
    //
    // The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
    //
    // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    // FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    // LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
    // IN THE SOFTWARE.
     
    using System.Reflection;
    using UnityEngine.Experimental.Rendering.Universal;
     
    namespace Game.Utils
    {
        /// <summary>
        /// It extends the Light2D class in order to be able to modify some private data members.
        /// </summary>
        public static class Light2DExtensions
        {
            /// <summary>
            /// Replaces the target sorting layers of the component.
            /// </summary>
            /// <param name="light">The object to modify.</param>
            /// <param name="sortingLayers">A list of sorting layer IDs to enable. Sorting layers not included in the list will be disabled.</param>
            public static void SetTargetSortingLayers(this Light2D light, int[] sortingLayers)
            {
                FieldInfo targetSortingLayersField = typeof(Light2D).GetField("m_ApplyToSortingLayers",
                                                                            BindingFlags.NonPublic |
                                                                            BindingFlags.Instance);
                targetSortingLayersField.SetValue(light, sortingLayers);
            }
        }
     
    }
// Copyright 2020 Alejandro Villalba Avila
//
// Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"),
// to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
// and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
// IN THE SOFTWARE.
 
using System.Reflection;
using UnityEngine.Experimental.Rendering.Universal;
 
namespace Game.Utils
{
 
    /// <summary>
    /// It extends the ShadowCaster2D class in order to be able to modify some private data members.
    /// </summary>
    public static class ShadowCaster2DExtensions
    {
        /// <summary>
        /// Replaces the target sorting layers of the component.
        /// </summary>
        /// <param name="shadowCaster">The object to modify.</param>
        /// <param name="sortingLayers">A list of sorting layer IDs to enable. Sorting layers not included in the list will be disabled.</param>
        public static void SetTargetSortingLayers(this ShadowCaster2D shadowCaster, int[] sortingLayers)
        {
            FieldInfo targetSortingLayersField = typeof(ShadowCaster2D).GetField("m_ApplyToSortingLayers",
                                                                        BindingFlags.NonPublic |
                                                                        BindingFlags.Instance);
            targetSortingLayersField.SetValue(shadowCaster, sortingLayers);
        }
}

Leave a Reply