Photo Credit: ludovic.celle

I spent some time recently thinking about the difference between override methods, which replace virtual methods (as far as outside callers are concerned), and new methods, which merely hide base class methods (not necessarily virtual methods either). While refactoring the other day I stumbled onto something I thought was clever, but that all hinges on whether shadowing and overriding behave the way I think they do.  So it seemed worthwhile to research it a bit.

After a simple Google search I found plenty of examples, but none that confirmed or denied all of my intuitions about the behavior of each method type.  As is often the case, it was more instructive to start a test project and explore the behavior myself.

Override Basics

Lets start with an example from csharpfaq.

public class Base
{
    public virtual void SomeMethod()
    {
    }
}

public class Derived : Base
{
    public override void SomeMethod()
    {
    }
}

That’s rather dull, we can jazz it up a bit.

public class BasicLogger
{
    protected virtual void WriteMessage(string source, string message)
    {
        EventLog.WriteEntry(source, message);
    }

    public void CallOutTheHour(string post, string hour)
    {
        this.WriteMessage(post, hour + " o'clock and all is well.");
    }
}

public class TraceLogger : BasicLogger
{
    private readonly TraceSource Log;
    public TraceLogger(TraceListener listener)
    {
        this.Log = new TraceSource("LoggerApp");
        this.Log.Switch.Level = SourceLevels.All;
        if (listener != null)
        {
            this.Log.Listeners.Add(listener);
        }
    }

    protected override void WriteMessage(string source, string message)
    {
        this.Log.TraceEvent(
            TraceEventType.Information, 
            0, 
            "{0}: {1}", 
            source,
            message);
    }
}

On versions of Windows that have UAC (Vista+ or 2008+) BasicLogger is basically untestable/unusable unless you want to run as administrator. That should make our tests clear, I should get an exception anytime BasicLogger.WriteMessage is called.

[TestClass]
public class ShadowsVsOverride
{
    [TestMethod]
    [ExpectedException(typeof(SecurityException))]
    public void TestBasicLogger()
    {
        new BasicLogger().CallOutTheHour("One", "Six");
    }
}

Likewise, we expect that TraceLogger instances will write out the message.

[TestMethod]
public void TestTraceLogger()
{
    using (var listener = new StringWriter())
    {
        new TraceLogger(new TextWriterTraceListener(listener))
            .CallOutTheHour("One", "Six");                
        Assert.AreEqual(
            "LoggerApp Information: 0 : One: Six o'clock and all is well." + Environment.NewLine, 
            listener.ToString());
    }
}

Note that CallOutTheHour is declared on the base, but that the WriteMessage method it actually calls is on the derived class. This is what people mean when they say that the virtual method is replaced, the base class can’t even call its own WriteMessage method. Once something is overridden, only the derived class has the option of calling the original method. We could write a method like this on TraceLogger:

public void WriteToEventLogDirect(string source, string message)
{
    base.WriteMessage(source, message);
}

We can see that this “works” by observing the exception.

[TestMethod]
[ExpectedException(typeof(SecurityException))]
public void TestWriteDirectlyToEventLog()
{
    new TraceLogger(null).WriteToEventLogDirect("Foo", "Bar");
}

To beat this to death we can see that it doesn’t matter what type the calling code thinks its using, the override replaces the virtual.

[TestMethod]
public void TestCasting()
{
    using (var listener = new StringWriter())
    {
        TraceLogger t = new TraceLogger(new TextWriterTraceListener(listener));
        BasicLogger target = t;

        target.CallOutTheHour("One", "Six");
        Assert.AreEqual(
            "LoggerApp Information: 0 : One: Six o'clock and all is well." + Environment.NewLine,
            listener.ToString());
    }
}

Shadowing Basics

Shadowing, using the Shadows keyword in Visual Basic, or the new keyword in C#, “merely hides” the method in the base class. What does this mean? Lets modify our loggers and see. First I’m going to make BasicLogger.WriteMessage a non-virtual method.

protected void WriteMessage(string source, string message)
{
    EventLog.WriteEntry(source, message);
}

This generates a compiler error:

'LoggerApp.Shadow.TraceLogger.WriteMessage(string, string)': cannot override inherited member 'LoggerApp.Shadow.BasicLogger.WriteMessage(string, string)' because it is not marked virtual, abstract, or override 

The Visual Basic compiler is even more to the point.

'Protected Overrides Sub WriteMessage(source As String, message As String)' cannot override 'Protected Sub WriteMessage(source As String, message As String)' because it is not declared 'Overridable' 

Although, in both languages “it” is ambiguous. We know that “it” refers to the base class method, because that’s all I changed.  We can resolve the error by removing the override declaration.

protected void WriteMessage(string source, string message)
{
    this.Log.TraceEvent(
        TraceEventType.Information, 
        0, 
        "{0}: {1}", 
        source,
        message);
}

Removing the override keyword in TraceLogger changes the error into a warning in C#.

'LoggerApp.Shadow.TraceLogger.WriteMessage(string, string)' hides inherited member 'LoggerApp.Shadow.BasicLogger.WriteMessage(string, string)'. Use the new keyword if hiding was intended. 

Visual Basic also generates a warning, but doesn’t suggest the Shadows keyword, instead it suggests Overloads.

sub 'WriteMessage' shadows an overloadable member declared in the base class 'BasicLogger'. If you want to overload the base method, this method must be declared 'Overloads'. 

I guess we would have to look at the IL to see if there is a difference between Shadows and Overloads in this situation, but I’m not going to go that far today. For now we’ll add the new keyword to our TraceLogger.WriteMessage declaration and see how our tests run.

protected new void WriteMessage(string source, string message)
{
    this.Log.TraceEvent(
        TraceEventType.Information, 
        0, 
        "{0}: {1}", 
        source,
        message);
}

When using shadowing, TestCasting and TestTraceLogger both fail with exceptions. In fact, every method now throws exceptions, its just that the other two tests were expecting exceptions, and these two were not. It seems now that BasicLogger.WriteMessage is always called. Before exploring why, I’ll decorate these two tests with ExpectedExceptionAttributes so that they pass, then we’ll write some new tests to continue to explore shadowing.

CallOutTheHour is declared on BasicLogger and so is “underneath” the shadow cast by TraceLogger. TraceLogger is doing its best to hide the method with its own implementation, but it can’t hide it from BasicLogger‘s own methods. However, from any code that recognizes TraceLogger as a TraceLogger (including TraceLogger itself), the shadow prevents the BaseLogger.WriteMessage method from being executed. Lets give TraceLogger its own public method to see that it can still call its own method:

public class TraceLogger : BasicLogger
{
    private readonly TraceSource Log;
    public TraceLogger(TraceListener listener)
    {
        // ...
    }

    protected new void WriteMessage(string source, string message)
    {
        // ...
    }

    // ...

    public void CallOutAlarm(string source, string message)
    {
        this.WriteMessage(source, "To Arms! It's " + message + "!");
    }
}

And here’s the test to verify:

[TestMethod]
public void TestSibling()
{
    using (var listener = new StringWriter())
    {
        new TraceLogger(new TextWriterTraceListener(listener))
            .CallOutAlarm("One", "Robin Hood");
        Assert.AreEqual(
            "LoggerApp Information: 0 : One: To Arms! It's Robin Hood!" + Environment.NewLine,
            listener.ToString());
    }
}

TraceLogger can still call the BaseLogger method using the base keyword. We don’t need a new test for this because our existing TestWriteDirectlyToEventLog test already covers that scenario.

Shadowing has an interesting capability which overriding lacks. You can use shadowing to change the visibility of a method. This lets us make WriteMessage public on TraceLogger if we like.

public new void WriteMessage(string source, string message) 

After this change, we have no errors or warnings, and our tests still pass. But now we can test WriteMessage directly if we want to.

[TestMethod]
public void TestWriteMessage()
{
    using (var listener = new StringWriter())
    {
         new TraceLogger(new TextWriterTraceListener(listener))
            .WriteMessage("Foo", "Bar");
        Assert.AreEqual(
            "LoggerApp Information: 0 : Foo: Bar" + Environment.NewLine,
            listener.ToString());
    }
}

This capability is interesting, but confusing. Compared to overriding, there is more mental baggage to keep track of, you have to remember what side of the shadow each piece of code is on, and you have to remember what interface the caller is using. Callers that create objects like this:

var t = new TraceLogger(...); TraceLogger v = new TraceLogger(...); 

Will get different behavior than those who declare the reference like this:

BasicLogger b = new TraceLogger(...); 

That’s something to think about before jumping into method hiding.

Considerations

Some people, when learning about shadowing, become very consternated. They think hiding methods is a bad idea, and it probably is. Then they go one step too far and use is as an example of bad design in .NET. But, it has nothing to do with .NET, C++ has always had the ability to hide non-virtual methods. Consider this code from SO#5289774

class Foo;
struct B : A<Foo> {
  void doSomething();
};

// and in the .cpp file:
#include "Foo.h"

void B::doSomething() {
  A<Foo>::doSomething();
}

Instead of just letting this happen, .NET has a keyword that lets you do it explicitly, and the compiler has a warning that might give you a clue that what you’re about to do is not what you intend. But the point in both languages is that hiding methods is a confusing implementation choice and should be used sparingly if at all. In simple cases like our logger, we should have just added another public method instead of using shadowing to increase the visibility of WriteMessage. If you only want increased visibility for testing purposes, you can subclass in your test project and place the extra accessor method there.