Solutions in Search of Problems: Disadvantages of Modular Designs

In software designs, developers often talk about "modular design," "separating concerns," and "loose couplings" as being good things.  The idea is that software should be built by identifying individual pieces of functionality, isolating each piece of functionality from the rest of the system, and then assembling the system out of pieces like building something with Lego.  Many developers and "architects" believe any other approach is wrong.  Indeed, nowadays tightly-coupled designs are usually wrong-ish, but not always.  Failure to recognize when to break the rules can result in a lot of unnecessary pain.

In the process of isolating individual pieces of functionality, useful features of that functionality are sometimes lost.  Perhaps the best examples of this is seen in DAL (Data Access Layer) abstractions which are intended to isolate the rest of the software from any particular database implementation.  The problem is that no DAL can cover all of the features of any particular DBMS (Data Base Management System).  Most of the time I've seen teams just act as if the the particular DBMS feature doesn't exist, or hope that the DAL vendor includes that feature in its next release.  Or, they create a custom DAL comprised of a hybrid of an off-the-shelf library like Entity Framework and custom code.  There are times when loose coupling makes perfect sense, like when you think it is reasonable that in the next 5 years you could change to a different vendor's database.  But on most projects, this isn't going to be the case.  And yet, a lot of .NET development teams intentionally handcuff themselves by shunning quite a few features offered by DBMSs that aren't abstracted by EF.   In many cases it would be more optimal to tightly bind your software to the DBMS the company has chosen and take full advantage of every single feature the DBMS offers.  How often do companies really drop SQL Server for Oracle or MySql anyway?  If you're migrating to MongoDb, EF is the least of your refactoring problems....

A simple example of this is hand-powered screw drivers.  Most of us have one of those screw driver sets where you can swap out a phillips-head for a flat-head or hex-head.  The software designer would say that the screw head and handle body are "loosely coupled" and that is a good thing.  But it isn't always a perfect thing!  Many of us have learned the hard way that while those modular screw drivers are good for light tightening jobs, they aren't great for heavy jobs where you need a dedicated fixed-head screw driver.  Especially if maybe you also need to use the screw driver for prying, hammering or chiseling, the modular sets are just too fragile, and screwdriver where the handle and bit are "tightly coupled" is the better choice for the application.  Right tool for the right job!

This lesson applies to all areas of software functionality, not just the database.  Sometimes there is a high likelihood of component changes, so abstraction makes sense, even if there is risk of component feature-loss.  But other times the cost in terms of feature loss is not worth it, and your software will perform better and be easier to maintain if some components are more tightly coupled.  It is all relative to your real world needs, not some abstract textbook.

I don't think there is any golden rule on this.  Perhaps a good rule of thumb is that if you can't articulate a reason why a particular component would change in the future, then the argument for abstracting it is less strong.  Every project has a budget, so focus your efforts on compartmentalizing pieces of your software that are the most likely to change in the future. I've seen too many projects where premature compartmentalizing got out of hand and resulted in a design that fixed problems that never came to exist.

Trust and Humility Keeps You out of the Box!

It never ceases to amaze me how susceptible we are to Groupthink.  Software developers compound the problem with that unique egoism they have.  If you want to be an "out of the box" thinker, it's important you not let yourself get pulled into the box by the Group.  This doesn't make you any less of a team player if you understand the dynamic.

I see this happen all the time on software projects:  two or more developers announce they are struggling with vexing problem X, and the rest of the team piles on, with each next developer thinking he or she is the sole genius who can solve X.  Many hours and emails and lines of code later, eventually someone figures out that all along the problem was really Y.  And worse, now in retrospect it seems so obvious.  How did the whole group miss it, and why was so much time and energy wasted chasing X?

Today in our office while I was on a call, there was a weird, annoying high pitched noise coming from someone's desk.  The office-mates came to investigate and help stop the noise.  Somehow it got planted in the group's collective mindset that the noise was coming from the printer.  They tried unplugging it, pressing buttons, swearing, casting spells, etc. all with no success.

When my call ended, I walked over to see if I could help somehow.  People kept yapping about the printer.  I initially had no idea if the problem was the printer or not.  But I did have enough faith in my teammates' abilities to realize that if it was the printer, they would figure that out with or without me.  The only help I could provide was to test their primary assumption.  There were a lot of devices on that desk, and my ears were not so certain it was the printer in the first place.  I started fiddling with the other devices on the desk, and ultimately realized that some clutter on the desk was pushing a button on the phone, and lo and behold it turned out the phone was the culprit.  Poor printer, nobody ever apologized to it.

I was able to avoid being drawn into the Box by simply trusting in my teammates abilities to solve problems.  If smart people were struggling with a problem, then maybe somehow they had been mislead as to the real problem.  I made a conscious decision not to accept their premises, and not step into their Box.

It is true that sometimes the problem really is X, just as the team initially believes.  And that means sometimes you will waste time and resources challenging the assumption of X and looking into whether the real problem is Y or Z, instead of just jumping into the Box.  But on good teams, in the long run, it is usually best to have someone look around the Box first before throwing another brain into the Box.

Random Annoyance: The C# Delegate Interview Question almost Nobody Gets

Part of my job is to interview other software developers, roughly 3-5 a week.  I hate to be "that guy" who nitpicks interview question responses.  Usually I am very patient and more interested with how candidates think, rather than what they happen to know today.  I try to work with that nervous person on the other end of the line to help them shine.  But one question I ask is almost never answered very well:

In C#, what is a delegate?

I find this question is answered in the following ways:

-15% "I don't know."
-5% "It's used for events but I don't remember more than that."
-79+% "It's like a function pointer like in [insert C or C-derived language here].  You use them for events."
-<1% "A delegate is a type that represents references to methods with a particular parameter list and return type.  You can do all kinds of things with them like event handling, multi-threading..."

Of the 80% that conceptualize C# delegates as pointer-ish, maybe 5% can answer what an anonymous delegate is.  Even a high percentage of developers that are using Linq methodically or lambda syntax generally don't recognize their own use of anonymous delegates.  When I probe with followup questions, I find it's not just an issue of not knowing the terminology but rather not really understanding conceptually what's happening.

I wouldn't say this question is necessarily a litmus test for me, but I feel strongly that questions like this tell me the depth of your "passion" for making the most of your tools.  How curious are you about the tools you have in your toolbox?  Are you constantly looking for new and creative ways to use those tools?  To abuse the 80/20 rule, sure--80% of the time just knowing a delegate is "like a pointer" and being able to use lambda syntax without understanding it will allow you to solve the problem and keep the project moving.  It's that 20% of the time (and probably much less than that) when really knowing what your tools are capable of is when you can shine and really produce an elegant solution and save the team a lot of time and grief.

Long story short, I've never not recommended hiring someone who answered my delegates question well.  As they say, RTFM ;-)

JavaScript setters and CustomEvent versus "Canned" Observables for MVVM

Most modern browsers now support JavaScript's setter syntax for objects as well as CustomEvent dispatches.  This opens the door to home-brew a simple observable pattern when, for whatever reason, a library offering "canned" observable patterns (like knockout.js or backbone.js) isn't available.

And why wouldn't an mvvm library be available?  In my experience, this can happen because a client or other decision maker rationally or irrationally forbids certain .js libraries.  Or maybe your page really is super-simple (they all start they way...) and doesn't warrant the overhead of full blown mvvm library.  Or...or...you want an mvvm framework to work exactly as you want it to...so you build your own!

First, a simple example of setter syntax, which allows you to act whenever something is assigned to a particular property of your object.  All we're going to do for now is log the occurrence:

var x = {
    _b:5,
    set b(myval) {
       console.log("b was assigned something!");
       this._b = myval;
    },
    get b() {
       return this._b;
    },
};

x.b = 7;
console.log("b is " + x.b);
console.log("_b is " + x._b);

Results:

b was assigned something!
b is 7
_b is 7

Now let's take it just a step further and raise and event at the document level for which we can listen and act:

var y = {_b:5,
set b(myval) {
    
this._b = myval;
document.dispatchEvent( new CustomEvent('b_set', { 'detail': this._b }));
  },
  get b() {return this._b;},
  
};

document.addEventListener('b_set',function(e) {
console.log("b_set event heard! b is : " + e.detail);
});

y.b=7;

Results:

b_set event heard! b is : 7

This basic pattern can be the foundation for an observable pattern that fits your particular situation.  There are a lot of creative things you can do with event names, the event detail object, custom element attributes, etc. to make your own little (or not so little) custom mvvm framework.

Adding Action to your .Add

I've learned not to be intimidated by C#'s rich library for delegate and lambda operations.  In fact, I've come to embrace Func and Action as tools to simplify APIs and keep my code clean.  One pattern I really liked when I first saw it, and try to encourage its use, is offering an override along the lines of .Add<T>(Action<T> aNewT) any time an API exposes the ability to add something to a collection.

The vendors of server-side web controls all use this pattern for adding columns to a grid control. First consider the delegate-less approach:

GridColumn myColumn = new GridColumn();

myColumn.Title = "Name";

myColumn.Sortable = true;

myColumn.Filterable = true;

etc...

mySuperGrid.Columns.Add(myColumn);

Maybe we can clean things up a little by declaring the new anonymous object within .Add method call:

mySuperGrid.Columns.Add(new GridColumn() 

{

      Title = "Name";

       etc...

});

But most of the time, won't I always want to add a new object?  So why make me bother to type "new"?  And if I can only add GridColumn types, why make me type that too?  So consider the following override for the .Add method:

public void Add(Action<GridColumn> initializerDelegate)

{

       GridColumn newColumn = new GridColumn();

       //We could even do some default intialization if it wasn't already done in GridColumn's 

       //constructor...

       newColumn.myProperty=myDefaultValue;

       

       initializerDelegate(newColumn);

       myColumnCollection.Add(newColumn);

}

Now look what a consumer of the API can do:

mySuperGrid.Columns.Add(column=>

{

      column.Title="Name";

      column.Sortable=true;

      etc..

});

In my opinion, this is much cleaner and simpler for consumers of the .Add method.  Note the consumer doesn't have to know the name of the type used to represent a new column, nor worry about correctly instantiating a new instance of one. Compile-time binding and Intellisense still apply!

So don't fear "=>", it really is your friend!

All is not Equals in JavaScript Frameworks!

Love it or hate it, a common pattern utilized by JavaScript framework libraries is to set properties of observable objects using explicit functions.  In some frameworks, using the equals sign (=) for assignment, although intuitive, will actually break stuff!  This can be a common source of nasty bugs if you're not careful.

If I were inclined to create or extend a framework, I would probably try to take the approach Angular.js uses and funnel developers towards a pattern where using the equals sign for assignment behaves as expected, i.e. observers immediately see the change.  I would offer a method whereby an unobserved change could intentionally be made.  So by default you could do this:

myModel.value = "New value!";

...and observers of myModel would be immediately notified.  If you wanted to hide the change for some reason, you could do:

myModel.suppressObservations("value","New value!");

...or something like that, but with a better method name ;-)

Here are some examples from frameworks I work with and how the equals sign can get you into trouble (note: JavaScript frameworks are constantly evolving.  Examples below could become obsolete quickly!):

A. knockout.js

Observable objects in knockout.js have properties that are set by treating each property as a method:

var myViewModel = {
    myString: ko.observable('Hello'),
}

To change the value of the myString property you must treat the property as a function:

myViewModel.myString('World!');

And something like this will actually "break" the observable because you'll be setting the property to a value instead of a function:

myViewModel.myString = 'World';

B. Kendo UI 

Observable objects in Kendo are a little more robust than in knockout.js and work like this:

var myViewModel = new kendo.data.ObservableObject({ myString: "Hello" });

To change the value of myString in Kendo:

myViewModel.set("myString","World!");

It is safe to use the equals sign, but observers won't immediately be informed of the change:

myViewModel.myString = "World!";

There are two ways to view this approach.  On the one hand, you can update a property value without triggering responses in observers, and maybe there are times you want to do that.  But on the other hand, it is pretty easy to create a bug by mistakenly using the equals sign for assignment and accidentally hiding changes from observers.

C. backbone.js

Models in backbone don't expose properties for manipulation by the equals sign:

var myViewModel = new Backbone.Model({
  myString: "Hello!";
});

To change the value in backbone:

myViewModel.set({myString: "World!"});

You can (mistakenly) do this:

myViewModel.myString = "World!";

I think this problem is a little more serious than Kendo UI's because myViewModel.myString is a completely different property than the one manipulated with myViewModel.set({myString: ... }) and myViewModel.get('myString').  Here it is not a matter of having the option of hiding a change from observers, and more likely creating a very subtle bug in your code.

D. Angular.js

Angular.js is a different beast altogether compare do the frameworks above.  Typically, you define your model as a plain old JavaScript object and manipulate it inside a scoped controller.  The most common pattern is to have the model expose its own methods for manipulation, and DOM elements are bound to the controller.  In other words, you will usually manipulate the model using the equals sign for assignment.

However, in cases where you do want to manipulate the object from outside (which might be considered bad practice anyway...) you deal with the object within the context of what Angular.js calls a 'scope' which exposes the .$apply function:

myScope.$apply(function() {
        myScope.myString ="World!";
});

Similar to Kendo UI, you can just do:

myScope.myString = "World!";

...but this goes unnoticed by observers and just like in Kendo UI you probably will rarely intend to do this.  I suspect most applications built on angular try to avoid .$apply and try to do all property manipulation within the controller using standard assignment with the equals sign.