Code structure – The House Metaphor

Reading code vs. writing code

In a legacy system, what is it that you spend absolutely most time on?

I would say, understanding code and finding out where to do a change. Most of us have been in the situation that we spend hours or days looking for exactly one single line that would be changed in a million line code base. And of course trying to figure out what effects that little change would get. Expected and unexpected.

In an old Cobol system I came across, even the most experienced developers often spent two weeks analysing what consequences a change would get. Of course that was before the time of unit tests and modularisation of code into microservices, but the correlation of the time it took to write code compared to change it was very obvious.

What are we looking for in a code base?

Let’s say I’ve got at task at my new job. I’ve got requirements that needs a change in the code. I have to find the exact lines to change. Being in this situation, I’m often confused. I can’t immediately see the overall structure and don’t know where to look for the code I’m about to change. I try to figure out certain things, for example:

Where is the code..

  • ..providing the UI or API?
  • ..that call the infrastructure like database or file system?
  • ..handling integrations with other systems or services?
  • ..handling the business logic?
  • ..defining configuration?

Also, I’m looking for the code that handles the business domain that I want do my changes in? In a monolith there might not be any separations by domains. In a microservice architecture there might be a better separation. For example, managing hotels is handled in the Hotel Management Service. Price changes are handled in the Pricing Service. And so on. Preferably I would like some form of graphical overview that could map the business to the system.

The house metaphor

When talking about this, my and my teammates often use the house metaphor. Many of us have owned or lived in a house, we know how they usually are structured and we get help from craftsmen to do maintenance and changes on our houses. There is a lot of similarities with a system.

A house is divided into separate rooms. Each room has a purpose; cooking, sleeping, storage, bathing and so on. Each room is optimised for the users’ (people who live there) needs.

Craftsmen come to the house with a mission to improve something or fix stuff that has broke. Depending on what they’re going to fix, they go to different rooms. They immediately know which room they are looking for. If they are about to install a new shower, they will go to the bathroom, if they are fixing the dishwasher machine they go to the kitchen. Nobody would look for a shower in the bedroom. There might of course occasionally be exceptions, I heard about someone having a TV built in to the bathroom floor.

If there isn’t a map, they will make a quick glance in each room to find out what type of rooms it is or ask the owner. Within a few minutes they grasp the location of everything in the house.

Create 2D Floor Plans easily with RoomSketcher. Draw yourself or order. Perfect for real estate, home design and office projects. High-quality for print & web.
roomsketcher.com

Before looking for the appropriate room, the craftsmen must of course find the correct house. In a city of millions of houses. First they might ask which district the house belongs to, and after that which address.

Here the craftsmen has an advantage compared to developers. We rarely get the “address” of the service we’re going to change together with the requirements. Sometimes we can start out in the UI and follow the code to find out in which are the changes should be done, and sometimes we simply have to ask someone in our team. Preferably, we always go through the changes with someone that is experienced within the system before we start to find out where to make the change.

What can we learn from managing houses?

If we could significantly decrease the time a developer spend on trying to find out what the system does, there would be a huge benefit for companies. Time could be spent on new features, and more features. So how do we do that?

  1. Define what it is you’re building. Is it a storehouse, a bungalow or a skyscraper? Sometimes you start with a bungalow and end up with a skyscraper, it’s pretty obvious that the architecture is not clear and easy to follow in that case. When this happens, refactoring is crucial to build a maintainable system.
  2. Define responsibilities. Define responsibilities of services (e.g. hotel management), layers (e.g. domain layer) and classes (e.g. controller, repository). Then developers immediately will understand where to look for the code they want to change. Is the change about the contract between the client and the API, then look in the controller classes in the API layer. Is it about the communication to the database, then look in the repository classes. The shower should be in the bathroom!
  3. Use commonly used patterns and guidelines. Most houses have a similar set of rooms, with similar purpose. This makes it easy for us to quickly understand the usage of each room. I your system follows a well-known way of structure the code, it will be easier for new users to start working with it.

And one more thing. Draw a picture of the system and make sure that the system’s file structure represents that picture. If there is a box named Hotel Management on the picture, make a folder named Hotel Management. If there is a layer named domain layer, make sure the developers can find a folder with that name in the code.

I think most developers find the above ideas pretty obvious. Despite that, many systems end up with an unclear structure. There are many reasons for that; tight time schedules, bad communication in the team, problems in combining helicopter perspective with frog perspective.

Conclusion

Systems often live up to 20 years and during that time a certain amount of developers are passing through. All of them with different experiences from previous systems they have worked with. If they could be up and running quickly, if they know what they are doing when making a change and if they easily can find a specific place in the code then time to delivery will be shortened and the quality improved. The changes will be more aligned with existing code and cause less confusion for other developers. The risk of causing bugs with the change will be lowered.

Companies will have everything to win, and nothing to lose, on encouraging the development team to keep a good code structure!

Same theme, another metaphor: https://medium.com/nick-tune-tech-strategy-blog/the-chocolate-sauce-design-heuristic-3cdc143dd2ea

Unit testing with F#

I recently heard about a project where the code was written in C#, but where they implemented the unit tests in F#. They did so because they wanted to learn and evaluate F# in a protected environment. I thought that sounded like a very good idea and wanted to try.

I picked the unit tests used in builder pattern blog post. My idea was that the builder pattern would be replaced by abilities built into the language itself. Let’s see if that is a correct assumption.

From Fluent to Composition

This C# code is taken from my builder pattern blog post. It dynamically builds an order to use in the unit test, and is written with a fluent syntax:

            var order = new OrderBuilder()
                .WithSingle()
                .WithStatusInProgress()
                .Build()
                .First();

But, how do we write fluent code in F#. The answer is, we don’t:

Now the concept of “fluent interfaces” and “method chaining” is really only relevant for object-oriented design. In a functional language like F#, the nearest equivalent would be the use of the pipeline operator to chain a set of functions together.

F# for fun and profit

F# is a composable language, using functions as building blocks. Composition can be done in different ways, with function pipelining (|>) or function composition (>>). By defining functions that has the same type as argument and return type, they can be chained together in a very clean and decent way. Let’s see how that works.

The C# model

The C# model that we will test from our F# tests is just a simple order class with two methods:

    public class Order
    {
        public OrderStatus Status { get; set; }

        public void Cancel ()
        {
            Status = OrderStatus.Canceled;
        }

        public void StartProcessing()
        {
            Status = OrderStatus.InProgress;
        }
    }

It is stateful and the methods return void. This type of classes can be written in many ways, but I think this is rather common so I’ll use it even though it’s not perfect for the F# code which we’ll see later.

Create the builder

The builder class in C# is stateful (see blog post referred above), because it has class variable to which each function adds data. Using class variables means that the methods has a side-effect, and side-effects are something that we’ve learned that we should avoid if possible.

The functions in the F# builder are very simple and with no side effects as they operate on their indata (or create new data). The first one creates the order itself and returns it. The next one takes an order as an argument, changes it and returns it:

let withSingle () = 
    Order()

let withStatusInProgress (order : Order) = 
    order.StartProcessing()
    order

In F# we don’t write “return”, the value of the last line is automatically returned. Order() is the same as “new Order()” in C#. Note that if the method “order.StartProcessing” had returned the order, we could have omitted the last line of the second function. But I wanted to keep the C# code as we usually write it, like it would be in a real case.

So, this is the builder for now. We’ll keep it simple and find out later how to handle lists (which we do in the C# builder).

Function pipelining

This is what a test can look like in F#, FsUnit is used to get the nice assert syntax. Here we use pipelining to combine different functions that produces and manipulates the order.

[<Fact>]
let ``Cancel order - pipe test`` () =
    
    //Arrange
    let order = 
        OrderBuilder.withSingle()
        |> OrderBuilder.withStatusInProgress

    //Act
    order.Cancel()

    //Assert
    order.Status |> should equal OrderStatus.Canceled

“OrderBuilder.withSingle” is called to get an order that is used as input the next function “OrderBuider.withStatusInProgress”. The order that is sent to, and received from the functions is not visible in the code. This might be confusing at first, but when you get used to it it’s nice because it keeps you code cleaner.

..the function parameters can often be ignored when doing function composition, which reduces visual clutter.

F# for fun and profit

Function composition

With function composition it’s possible to combine functions in many different ways, and store them in variables. These function variables can then be combined with each other. If we use function composition in our test it would look like this instead:

[<Fact>]
let ``Cancel order composition test`` () =
    
    //Arrange
    let createOrder = 
        OrderBuilder.withSingle 
        >> OrderBuilder.withStatusInProgress

    let order = createOrder()

    //Act
    order.Cancel()

    //Assert
    order.Status |> should equal OrderStatus.Canceled

In this test there is an extra line to actually run the function. The best written tests should be both readable and short, so probably pipelining would be the best choice in this specific example.

Build lists

How do we build lists in a clean and simple way in our tests? First let’s add another function to our builder, “withStatusCancelled”:

let withSingle () = 
    Order()

let withStatusInProgress (order : Order) = 
    order.StartProcessing()
    order

let withStatusCancelled (order : Order) = 
    order.Cancel()
    order

Let’s say the test should build a list with one order that is in progress, and one that is cancelled. By using composition, two functions are defined by combining two other functions:

    //Arrange
    let createInProgressOrder = OrderBuilder.withSingle >> OrderBuilder.withStatusInProgress
    let createCancelledOrder = OrderBuilder.withSingle >> OrderBuilder.withStatusCancelled
    
    let orders = [createInProgressOrder(); createCancelledOrder()] 

To make the code even more compact, the following can be done without losing readability:

    //Arrange
    let orders = [
        (OrderBuilder.withSingle >> OrderBuilder.withStatusInProgress)()
        (OrderBuilder.withSingle >> OrderBuilder.withStatusCancelled)()
    ]

What do you think? Which one is the best? Or does anyone have a better suggestion?

Conclusion

Since F# is a language that is built for composition, it’s great for providing flexible data to tests. The amount of code needed for the OrderBuilder is significantly reduced compared to in C#. Handling lists is very easy and can be kept outside the builder, in the test itself.

Using F# for the unit tests is not only a good and safe way to get used to a functional language. It also makes the tests simpler and reduce boilerplate and the number of lines in your code.

After getting used to it, I also think that it makes the code more readable! (If written in a good way, but that applies to C# too). I recommend you to read more about how to use it as a Domain-Specific Language to take this one step further!


Download the complete code here.

Confused? Want to learn more?
Spend 60 seconds to better understand F#
Want to learn more about building blocks?


Letter to the coding newbie

Dear Newbie,

I know you struggle a bit with your new role and how to become a successful professional programmer. I think the best would be if I just told you, already now at start, what you need to know!

Many newbies are worried that they are not smart enough, but this is rarely the problem. What I’ve seen though, is lack of communication and humbleness. So, when you get stuck, don’t wait too long before you ask. Just do it! Asking questions means you’re smart! And even smarter is to ask the one in the team who you suspect might criticize your code later!

One advantage of being a newbie, is that you’re not supposed to know so much. I mean, after all, you’re a newbie! So instead of pretending to know stuff, be curious and try to learn all the time, every day. Being a person who wants to learn, is always appreciated by others. You’ll show the team that you’re genuine and you will learn and become more productive.

If the customer is happy, then I’m happy, you may think. And while this is of course true in one sense, it’s not the whole story. Because almost equally important is being a good team member. If the team agrees about something, you need to comply. Make friends with the team members and if something seems important to them try to work in that direction too. They might know something that you haven’t understood yet. Assume you work with smart people and try not to be the smartest person in the room.

I’m very glad that you’ve chosen this career path and I promise you a lot of great moments! Even if we old dinosaurs seems stressed sometimes, we love your enthusiasm and drive, and to relive memories from the time when we were newbies ourselves!

Yours sincerely,

Christina

Unit testing – Provide flexible data

It’s rather easy to find information about how to write a good unit test in the means of the test itself. But how should we handle the data that we need to test? How can we create test data structures in a flexible way?

As a DDD (Domain-Driven Development) person, I like to be able to read the code as I read a book. This is of course very hard, but something that I strive for. Therefore I often use the builder pattern for unit tests.

Builder pattern

Generally, a good approach is to create the data within the test itself. This makes it easy to see exactly what the test do. But after some tests you often find the same code repeated many times so you start looking for a generic way of generating the data suitable for each little thing you want to test. The builder pattern comes in handy here.

Let’s say you want to test the following test case:

* When you cancel an order, it gets the status cancelled. *

The test

You need to create an order with the status InProgress. Your test will then change the order to Canceled an verify that it worked. In a real system, you would probably have a lot of tests around status changes. Therefore it would be suitable to have code that creates an order in different statuses.

With the builder pattern to generate the order the test might look something like this:

        [Fact]
        public void CancelOrderTest()
        {
            //Arrange
            var order = new OrderBuilder()
                .WithSingle()
                .WithStatusInProgress()
                .Build()
                .First();

            //Act
            order.Cancel();
            
            //Assert
            order.Status.Should().Be(OrderStatus.Canceled);
        }

Let’s start with the “Arrange” part. A new OrderBuilder is created and asked to create a single order with the status InProgress. When looking into the builder later on you’ll see why the .Build() and .First() methods are needed.

The “Act” part here is intended to cancel the order, and the “Assert” checks whether the order has got the status Canceled. The FluentAssertions package is used to get a nice and fluent syntax of the assertions.

The builder

The OrderBuilder is a class with methods that returns the class itself. This is what makes it possible to use the fluent syntax in the test. Only the Build() method returns the actual list of orders:

    public class OrderBuilder
    {
        private List<Order> _orders = new List<Order>();

        public OrderBuilder WithSingle()
        {
            _orders.Add(new Order());
            return this;
        }

        public OrderBuilder WithStatusInProgress()
        {
            _orders.Last().StartProcessing();
            return this;
        }

        public List<Order> Build()
        {
            return _orders;
        }
    }

I chose to prepare the builder to handle many orders, by adding a private list variable in the top of it. Of course that could also be a single order.

The WithSingle and WithStatusInProgress methods both operates on the private class variable and step by step build up the order that the caller of the builder wants to have. More methods can be added and freely combined and called after each other to e.g. build up a list of orders:

        //Arrange
        var orders = new OrderBuilder()
            .WithSingle().WithStatusInProgress()
            .WithSingle().WithStatusNew()
            .Build();

The Build method returns the orders and breaks the chain by not returning the TestBuilder class.

A functional approach

Another approach to the problem is to use a functional way of building up the test data. Sometimes you have a lot of properties, and want to change them in a controlled way to get exact the data you want to test. You can add a method to the builder that lets you create an order this way:

    [Fact]
    public void CancelOrderTest2()
    {
        //Arrange
        var order = new OrderBuilder()
            .WithSingle(o => o.Status = OrderStatus.InProgress)
            .Build()
            .First();

        //Act
        order.Cancel();

        //Assert
        order.Status.Should().Be(OrderStatus.Canceled);
    }

You can from the test set which properties you want to change on the order, and combine with other builder methods if you want. The WithSingle builder method that takes a function parameter looks like this:

    internal OrderBuilder WithSingle(Action<Order> action)
    {
        _orders.Add(new Order());

        action?.Invoke(_orders.Last());

        return this;
    }

It takes a function as an argument, creates an order and applies the function to it.

Conclusion

The builder pattern has helped me a lot and it is always fun to use it! I show it here as I learned it from start, but in every project I use it in a special flavor suitable exactly for that special environment.

Functional domain modelling

Most of us have an idea of what functional programming is, and have at least used some of the functional features of our best known programming language such as Lamba or LINQ in .Net.

When I’ve tried to learn more about functional programming, I’ve always felt that something is missing. There is a lot focus on the functions themselves, but less on where to put them in a structure. I’ve asked fellow programmers, but even if they seem very confident in writing functions, they get quite fuzzy when it comes to implementation of a real business problem, and how to structure the code.

Since I’ve used reactjs for many years I have one good example of how to implement a functional code design. But I have troubles applying these principles to the server side, or to see them as a more common approach. So I applied to a workshop at DDDEurope: Lean and functional modelling, with Marcello Duarte. This is what I learned:

You have to start with the programming

To understand the functional design you need to start with the programming. By understanding some of the most fundamental concepts in functional programming it will be easier to grasp the modelling part later. Now, I’ll probably not be the best to explain this after just two days of learning, but I can give you an overview of what you need to dig into. We used Scala for the programming exercises, but I think this is something that you can find in several languages and some of it maybe even in C#.

I found some descriptions with examples in Javascript, which might be easier to grasp for most of us:

Higher order function: “A higher order function is a function that takes a function as an argument, or returns a function. Higher order function is in contrast to first order functions, which don’t take a function as an argument or return a function as output”. Higher order functions.

Composition and currying: “A curried function is a function that takes multiple arguments one at a time. Given a function with 3 parameters, the curried version will take one argument and return a function that takes the next argument, which returns a function that takes the third argument. The last function returns the result of applying the function to all of its arguments.”
Curry and function composition

Functor: “A functor supplies a box with zero or more things inside, and a mapping interface. An array is a good example of a functor, but many other kinds of objects can be mapped over as well, including promises, streams, trees, objects, etc. JavaScript’s built in array and promise objects act like functors.”
Functors and Categories

Monads: “A monad is a way of composing functions that require context in addition to the return value, such as computation, branching, or I/O. Monads type lift, flatten and map so that the types line up for lifting functions a => M(b), making them composable. It’s a mapping from some type a to some type b along with some computational context, hidden in the implementation details of lift, flatten, and map.”
Monads made simple

Abstractions and compositions: “Software solutions should be decomposable into their component parts, and recomposable into new solutions, without changing the internal component implementation details.”
Abstraction and composition

Thanks to Eric Elliot for the descriptions and examples above!

When trying to write clean code there are things that I sometimes stumble on, e.g. keeping the number of parameters down and writing pure functions without side effects. To use a more functional approach is making it easier to write clean code.

Functional modelling

Functional architecture and design is often based on events and event-driven development. Concepts like event sourcing and CQRS can be considered as functional architectures. These architectures can of course be implemented (and is implemented) with any programming language and a wide range of technologies, but functional programming is very well suited for implementing this kind of architecture.

With event-driven development you identify a business flow and let the system reflect that flow. Data might change shape during this flow; for example in a purchasing domain you talk about a price of a product you buy, but in the finance domain the same amount is considered a cost of the same product. It is not just a different status of the product; it is considered a totally different thing.

Event Storming

A common way of identifying the business flow, is to do an event storming. At the event storming business events are identified, and also roles that are involved in firing the events, and the data that is sent. The team can do the event storming by themselves, but if possible, it is good to involve the business. The events, roles and data is written on post-its and put on the wall to illustrate the flow.

During the event storming it is important to avoid too much talk about implementation details (which is hard), and use cases (also hard). Try to focus on how the business work, the flow, and why things are done in a certain way. Look out for the core domain, and try to understand how it contributes to the business. It is important to understand what the business makes money of!

When the flow is defined it is possible to identify the services. One service should reflect one part of the business (domain). The data flows between the services in form of events, and very often the user roles indicate where it is suitable to draw the boundary of the service:

The services can be implemented in many different ways and they have their own databases or data stores. E.g. one is using a SQL database, while the other is using Eventstore and a CQRS approach. Since the services in this example is part of the same deployable unit there are some limitations when it comes to choosing languages and technology. Each service is responsible for its business rules, and don’t know anything about the others. Each service owns its own data.

Separate Responsibilities

In functional programming it is crucial to separate responsibilities into different parts of the code. This has spread to the object oriented systems too, so for many of us this comes very natural. The business logic is considered as the core of the system, and if you change technical implementations around it (database, client api’s etc.), it should be possible to leave the business logic untouched.

The flow throughout the system

The business logic implementation is composed of many independent functions which are called in a fluent way. I like to think of a system in the form of streams where the data flow and via projections and usage of different functions the data take different forms depending on what is should be used for. The flow sometimes temporarily stop waiting for input and is then saved in queues or databases. With event sourcing the flow could be reset and replayed at any time.

As a simple example, think about using LINQ or Lambda for filtering and mapping the the data that flows throughout the system. Then you’ve taken a big step. But try to think beyond that; can your own functions be written and applied to the flow in an even better way? Maybe, if we learn how to use the monads, functors etc. that was described earlier in this text.

Immutability

As an object oriented programmer I have some difficulties to stop thinking in objects with methods and properties. One problem with objects is that the functions often have side-effects, such as reading or modifying a property. Then the outcome of that function might not always be the same, with the same in-data. Side-effects must be avoided to make the functions composable.

The data that shall be modified in a function must be sent in as a variable, be recreated with the changes applied, and then sent back as the result of the function. The data is immutable. For those of us that has worked with reactjs, we’re quite familiar with this.

Conclusion

To fully understand functional programming is not easy. How to design composable functions using higher order functions, functors and monads is quite hard to grasp. On the other hand, functional programming has influenced us a lot already, and given us the thinking of responsibility, events, event sourcing, and event-driven design. Many of us are eager to learn how to use the reduce function in Javascript, how to implement reactjs in a better way, or writing LINQ and Lambda in C# and Java. All that work takes us closer to the paradigm shift toward functional programming.