Wednesday, September 17, 2014

Why you SHOULD use layered architecture

This post argues against using layered architecture since according to the post you simply don't change implementation of your database or implementation of JPA.

Firstly, the term layering is a misused term, layered architecture should be regarded as layered on its own. There are several different layers, such as communication, framework, hardware each of these might contain several other layers of multiple layers of some notion, such as a OS hosting processes etc. Layering is a code standpoint when you choose the "what" of all of the combined layers and present functionality and why they matter to the layer above.

Now secondly, if layers matters, you are doing layers wrong (layering to hide implementation of frameworks in a application which is about e-commerce is silly). Layering is organizing context, a common misunderstanding is that layering are abstractions. It has a lot to do with abstractions and most of the times its get confused with it, but it really is about context relaying, where the details of a function effects the surrounding code and layers.

Context relaying is heavily bound to the fact that abstractions leak. Some sort of combined function will always leak the inherent functionality. If it were true that the function wouldn't leak, it has to be so general that it will accept anything and reply with precisely what you want, i.e the God Service.

Now computers are inherently stupid, they will do as exactly as told even if its wrong, as long you talk its language. Problem is that whenever you are producing code you mostly focus on the how not the what. As an example is the inherently problematic behavior when writing objects. You might name an object as Animal and when you read this it seems like a very sensible thing to name your object because it might have a place in the object community. However this is blatantly wrong because the notion of Animal doesn't mean anything unless you provide proper context which is described with code, either directly or indirectly. Try switch the name of that object to X4 and things starts to make less sense. In fact, try obfuscating the whole codebase and try to understand what is going on, and if you do you might be closer to a good layering structure.

Now a layer is where a bunch of objects are making sense regardless of other bunches of objects are doing, even if there seems to be some sort of dependency of them. The code has to be reflecting this, if more than one "bunch" of code is breaking layers you have done layering wrong. Object oriented programming and TDD may produce bad layering because you are only are regarding the object or the"unit". To get this correct you have to focus on context and behaviour while creating the object and "unit". Both SOLID and the principle behind TDD encourages doing layering correctly if you focus on context and behaviour, combine them with context relaying you are one step closer to doing layering correctly.

Another factor of the "mess of layers" is the misuse of interfaces. And this is inherited by the fact that  its simple to produce code interfaces but hard to create data interfaces and those are mostly overlooked. Such as when creating simple CRUDs.

The idea of managing code with organizational structures

Recently Mckinsey published an article about how to achieving success in large complex software projects and their conclusion is cross-functional team produce better results. They mention an example of an insurer which had problems and switched to cross-functional teams. This let the individual programmers started to "focus on delivering end-to-end functionality rather than just thinking about their own roles" and because of this it communication improved, resolving requirements and solved problems faster. As a result code defects fell by 45% and 20% faster time to market.

The report mentions "agile, lean and test-driven" and shorter iterations but not exact methodology, but they do confine the development process into a team per module, whatever a module is defined as. Which should be aligned with having a cross-functional team.

However there are no, as usual, no mention of how the code were written, language used, just that you have to manage people and it will auto magically be a success. I think this mentality is really weird and typical for this kind of research. Also the example seems to be a new product which is defined from scratch, which means that most of the "bumps" is avoided (a legacy system is not documented and/or requires changes, rebuilding stuff because they didn't thought of your project when building things). As a conclusion, how you write the code really doesn't matter for your projects success as long they are producing functionality which the customer wants, and present belief, continuously asking the customer seems the way to go.

The claim that structuring your team which maps to the produced "modules" is interesting. Conway's law dictates that any system is doomed to be a direct copy of your organization. This has been dubbed to the Mirror thesis and the result is quite interesting. According to this, and somewhat Mckinsey, structure in the organisation should be reflected in the system. Two organizations which has taken this to an extreme is Avanza and Netflix. Avanza has apparently taken the scrum idea into the organization and utilizes it all the way up to the company board. Netflix tried to reverse Conway's law by implementing microservices and structure their organization from which architecture they wanted.

So if this is true having an agile organisation should produce an agile system. This should also be reflected in the code since the code outlines the system. So the thought here is that, in any given code base one should be able to get which type of organizational structure were used when building the software. Now if this is mirrored accordingly to which modules are produced which is probably fine, however if the modules starts to blend (more and more interdependencies between the modules, business grows, initial structure was off etc) with each other, or worse, the organization or business changes, due to external factors.

Now things tend to break apart, sometimes because its not feasible to change the codebase because theres no time or its not just understood. It might be so, combining information in new ways creates new interesting business areas which breaks old module "barriers". Management has no idea that a decision completely wrecks a structure, or system is simple not able to deal with the modular idea. Either way there's no way of escaping Conway's law, even with all the good intentions in the beginning.

But using an agile organization or development process (which is part of the organization) the code should be reflecting this somehow. The only evidence I can see is that TDD could produce "test induced design damage" which happens when testability becomes the main goal, and is not a good thing. The result will and always will be a result of your organizations communication "routes" since that what systems are all about, automatization of whatever case you have, and any deviation from that will cause an mismatch in your total organization and will create tearing problems.