DEV Community: CTO UK - West Midlands

Why product development is the beautiful game

rich_marshall — Fri, 21 May 2021 08:15:21 +0000

Building Teams

Firstly, I need to apologise to any football fans out there. I’m the first to admit I don’t follow the “Beautiful Game” but I keep turning back to it as a useful example. I could use many other team sports as a comparison instead but football seems to be the most universal.

One of the greatest challenges and in all honesty, frustrations, I’ve experienced over the past years is the treatment of highly skilled, talented, creative people as “resources”. I can’t begin to count the number of times a financial director or project manager has asked, suggested or requested “resources to be re-assigned, so a project goes faster”.

So much of this thinking dates back to the emergence of heavy-duty industrial and civil engineering projects where dockworkers would line up for selection in the morning and the one’s who could set the most rivets would be chosen that day. There is an increasing body of evidence and library of resources that demonstrate that building software products, especially those delivered in through a SaaS model, needs to be planed and managed very differently - I won’t go into the details of that here, but if you’re interested I can’t recommend Sooner, Safer, Happier enough.

It was during one of these many discussion that I was searching for a way to describe what they were requesting and a football team came to mind.

You’ll have to bear with me, because as I said, I don’t follow football but I know for a fact there have been many examples of where “super teams” have failed. It doesn’t matter if you put the worlds 11 best players in the same team; you’re not guaranteed success. More likely you’ll get 11 individuals, all of whom want to prove they’re the best.

The role of the manager is well understood in football, as is that of coach and captain. In fact all these roles are recognised as being vital to the success of a team and yet in software development, (from the outside, at least) these roles are often absent or seriously undervalued.

It’s also true to say that you can’t simply move a player from another team and expect instant success. Teams are carefully selected from squads and each player within that team is selected for the role they need to play and hw their skills and talents benefit the team. Managers will also pay attention to how any player’s deficits might affects others - people always have both good and bad traits.

I should all emphasise the value of cross-functional teams over silos squads of developers. Teams need to have all the skills in them, in order to be successful and you’re unlikely to find all of those skills in one person. For a football team to be successful, you need strikers, midfield, defenders and goal keepers - not to mention the coach, captain and manager roles. Product development teams are the same. 11 software developers is not going to solve the problem. You need product managers, designers, and often other subject matter experts in there too. Yes, you can get them on loan in the same way you can bring a substitute on to the pitch, but they still need to be part of the squad and know the team inside out.

I’m pretty happy with this analogy so far but how much can I push it…!? The rules of football dictate 11 people on the pitch from one team. So you can’t just keep adding more people to win. Product development teams don’;t have the same rules applied - at least officially, but there is significant evidence (start with Dunbar’s number and go from there) that there is a practical upper limit to the size of a high performing team and adding more people doesn’t improve performance but often makes things worse. If you tried putting 100 players on the pitch everyone would trip over each other!  There’s also a lot to be said about how a team actually work together. It’s still common practice to attend a standup and hear from each individual what they were working on and their progress, their blockers. It’s routine to see a team have each developer working on a different problem and then request support from their peers, who are in a different context. Some might see this is giving a football team a ball each - more opportunity to score a goal! But that’s far from the actual outcome. It also increases the likelihood of an own goal and who are you going to pass to when you get tackled. In fact, it’s not even like that! It’s more akin to expecting each individual player to be on a different pitch, playing a different game and we all know how that’s going to work out.

I’m pleased to say there is a growing body of evidence to shift thinking but it’s high time that we and our peers stop thinking of software engineers as being heavy engineering. That we stop using methods and languages that describe what we produce as projects like railway lines, with the idea that it’s finished when it reaches it’s destination and we can leave it for the next 100 years and most importantly, that we stop treating people with thoughts, ideas and passion as faceless resources, to be dragged and dropped at will where ever there’s a problem of a task to be completed.

Instead we need to think of our products and services as more akin to the service that runs on the railways and or teams as complex, ever evolving people who need support, coaching and management to work most effectively.

Sure, you can keep moving players from one team to the next or put a midfielder in goal or even send every player to have a different game but you’re not going to win the cup or the league that way and you’re probably going to get relegated pretty quickly if you try it.

Header image courtesy Chris Leipelt

'I wonder if...': Moving ASCII art

David Maidment — Fri, 14 May 2021 06:52:17 +0000

'ASCII cinema,' was all that I heard. Sat in a weekly sprint meeting, my mind focused more on arranging my upcoming week of work than paying attention to the person offering a retrospective of their just-completed sprint, my ears perked up at the mention of what sounded like something magical.

The reference was, of course, to asciinema, a tool for recording terminal sessions that was wholly-relevant to the retrospective. But all I heard was, 'You could play The Godfather in the terminal.'

Such technology speaks for itself, surely?

We all joked about my absentmindedness and mused for a few minutes on the practicalities of playing back video as moving ASCII art, then continued with the retrospective. But my brain couldn't let go of the challenge; by the end of the meeting I had already mapped out the basic workings of the codec in my mind's IDE.

For those as lazy as me who just want to see the end result, this is what I managed to code in the space of about half an hour, using a GIF as input, JSON as the storage format and Golang as the programming language:

Make it so

Having decided to write the most [un]necessary codec I could think of, my first thought was how to store the data.

In the spirit of misusing technology, I decided that JSON would be the way to go. The format certainly lent itself to storing necessary metadata such as the frame rate, which I planned to preserve from the input file and match on playback with dynamically calculated sleep commands.

I recently gained some experience working with images on the pixel level for another project, so my go-to approach was to read in an image (a GIF version of a movie seemed to make sense) and grab the RGB value of each pixel to save losslessly in some manner that could later be displayed in the terminal.

To begin with, I knew that I needed to store an initial key frame containing the colour values of all of the first frame's pixels. Coding in Golang, a map of maps of integers (an associative multidimensional array, in other languages) seemed to fit the bill, where the map keys would represent x/y pixel coordinates and the leaf values would be -1, 0 or 1 to denote the pixel's colour—I initially opted for a black / grey / white colour palette.

The JSON for such a key frame would look something like:

{
  "1":           // Row 1
    {
      "1": 1,    // Column 1
      "2": 0,    // Column 2
      "3": -1    // Column 3
    },
  "2":           // Row 2
    {
      "1": 0,    // Column 1
      "2": 0,    // Column 2
      "3": 0     // Column 3
    },
  "3":           // Row 3
    {
      "1": -1,   // Column 1
      "2": 1,    // Column 2
      "3": 1     // Column 3
    }
}

When the values for such a data set are arranged in order, a basic picture emerges:

00000000000000
00001111110000
00011111111000
00111111111100
00011111111000
00011111111000
00001111110000
00000000000000

Subsequent frames would also have their pixels traversed, but only those pixels whose values differed from the previous frame would be stored on the next frame map. For example:

{
  "1":           // Row 1
    {
      "2": 1,    // Column 2
      "3": 0     // Column 3
    },
  "3":           // Row 3
    {
      "3": -1    // Column 3
    }
}

If the only change in a frame was that a character blinked, then only data for the pixels around their eyes would need to be stored.

Playing it all back

As mentioned previously, I opted for black, white and grey as the three colours of the palette. My theory was that you could get a fairly good image out of those three colours by taking the average of each pixel's RGB components, ((R+G+B)/3), and assigning the lower third to black, the middle third to grey and the upper third to white.

Characters that covered an appropriate amount of space on-screen could be used to represent those colours (I chose a space, a slash and a capital W, respectively). Displaying them would simply involve building up multiple strings (one per scanline), flushing them all to the terminal once per frame and then issuing sleep and clear commands.

The first test produced this:

Three 'colours' worked, but not well enough.

The result wasn't terrible, but the image also wasn't as clear as I had hoped. I needed to preserve more colour data. After some googling I managed to find a list of around 70 characters that covered a wide range of screen real estate.

I processed my test video again but the result was disappointing—the animation was mostly recognisable, but the resolution was simply too low to make the extra 'colours' worth it.

After much tinkering, a selection of ten characters seemed to yield the best results. This is what you saw in the first GIF.

Improvements I should have made

There were some obvious improvements that I could have made to the codec, had I not given up once I had proved to myself that, yes, I could watch The Godfather in the terminal, if I really wanted to.

Experimenting with a 500MiB JSON file (converted from a 20-minute TV show), I was faced with a lead time of around one minute while the data was loaded into memory. It strikes me that abandoning JSON and switching to a one-frame-per-line format would enable streaming to begin instantly.

Switching to a non-JSON format would probably also yield lower file sizes, perhaps as small as 200MiB in this instance, assuming that low-impact delimiters were used in favour of the quotes and braces that comprise a sizeable percentage of JSON strings.

It would also be interesting to explore run-length encoding for key frames (and/or those frames with over 50% of pixel values changed). Although I have not yet carried out any experiments to confirm it, I suspect that the small 'colour' palette would lend itself nicely to this kind of compression.

Should you build it, or abandon it?

David Maidment — Thu, 29 Apr 2021 21:28:43 +0000

Over time a product will accumulate many features. Some are core features. Some are proofs of concept that have yet to be validated. Some are proofs of concept that failed. Some were once extremely relevant but are no longer fit for purpose. Some are a work in progress.

The point is: features accumulate, and when it comes to determining their relevance and maintainability, no two are the same. But one thing most features do have in common is that, at some point in time, someone thought they were a good idea.

We've all maintained poorly designed features. If we've been lucky (or maybe unlucky, depending on how you look at it), someone else was responsible for them. We've all looked at something and thought, 'Who built this, and what were they smoking?'

Hindsight is a wonderful thing.

And so the question presents itself: when confronted with the option of building or not building, of maintaining or abandoning, how should you proceed? How can you avoid becoming the guilty party at the other end of the 'What were they smoking?' jibe?

This question is particularly pertinent for early-stage companies, where resources are considerably more finite and the potential impact of every feature (positive or negative) is considerably larger.

It can be helpful to take a step back and answer two simple questions as bluntly as possible:

Did users ask for it?
Are users actually using it?

Or, if the feature has yet to be built:

Have users asked for it?
Have we validated that they actually need it?

The answers to those questions can help you categorise the current state of the feature; it may be: Mission Critical, a Stroke of Genius, a Waste of Time or a reminder that, often, Vision ≠ Reality.

The obvious takeaway may be: Spend most of your time building Mission Critical features and have game-changing innovations. However, it is important to understand that other types of features will end up in your product and to be realistic about your ability to consistently innovate.

By understanding the qualities of each quadrant, and how a feature may be masquerading as a different type, this process becomes easier.

Mission Critical features

These are the features your users have asked for, you have duly built and everyone uses. They are the bread and butter of your business and should be no-brainers. If you are a car manufacturer, these features are engines, airbags and steering wheels.

The decision to build these features comes from a combination of thorough market research, common sense and (if you are doing things right) dogfooding your own product.

However, as obvious as these features seem, it can still be worth challenging your assumptions about them. Especially if they are the product of well established received wisdom.

Do they represent an extremely ineffective way of doing things that your users have come to accept as an inevitable part of life? This is a hard question to answer, as it requires really thinking outside the box.

If the answer is 'yes', then also ask yourself whether an opportunity exists to remove the feature (it may actually be a Waste of Time that people have become accustomed to) or, even better, make a radical departure from the norm and disrupt the status quo by evolving it into a Stroke of Genius.

As the quote often attributed to Henry Ford goes:

'If I had asked people what they wanted, they would have said faster horses.'

Stroke of Genius features

When you shake up the industry and create something no one realised they needed, you have a Stroke of Genius on your hands.

Features (or often, entire products) that fall into this category usually take two forms: entirely new inventions, and things that are suddenly possible because of a larger paradigm shift brought on by those new inventions.

Most of our innovations will not fall into the new inventions category. As much as we like to think that our idea will change the world, very few ever do. So it helps to be realistic about the scope of whatever you are building.

Much of what we build instead rides the waves of new inventions. At the moment, the invention that many products and features piggyback off is the Internet. Many industries have been disrupted in a way that no one ever thought possible, largely down to the new abilities the Internet brought with it. Banking, shopping, photo processing, TV and film are just a few industries that now operate in entirely new ways.

So apply that thinking to your own product. Maybe you'd love to build a widget that helps to automate people's lives. The technology to build it has always been science fiction (maybe some kind of advanced AI?), and no one has asked for it because humans have been successfully living their lives for hundreds of thousands of years without it. Then suddenly the technology to build it becomes available; you're ready to change the world.

Simple, right?

Perhaps not. With a product no one has asked for, you will find yourself fighting against years, decades or centuries of established status quo. You'll have to convince everyone that they've been doing it wrong all this time. And no one likes to be proven wrong.

Just because you build it, it doesn't necessarily follow that they will come. You may even find building the feature to be the easiest part of the process, with huge amounts of time, effort and money required to run a successful awareness campaign to obtain even your first few hesitant users (electric cars might be a good example of this). This is where a Stroke of Genius risks becoming a Waste of Time.

To combat this, aim to work in MVPs and spend the least time possible to get a workable version of an unvalidated idea out into the world. Take feedback, iterate, but don't commit huge quantities of time to it. By operating in this way you can run far more high-potential experiments in a given period of time and will feel less guilty about axing one of them when the numbers don't add up.

It is also worth keeping perspective on the uniqueness of your idea. If it's been possible for a while to build your feature and no one in the market is currently doing so, this may not be an indication that you have a unique idea, but instead that it is a really bad idea that others have already tried and failed at. Take this into consideration when deciding how much time you're willing to sink into an MVP.

Waste of Time features

Your users haven't asked for the feature, you built it anyway and they aren't using it. This is a waste of your time and resources, and such features will always benefit from being put on hold.

A Waste of Time may be an unrealised Stroke of Genius, but probably not. Even if you are certain that you are onto something—something visionary and years ahead of its time—it is still worth limiting the time you spend on these features while you carry out market research (which may take the form of user feedback loops) and explore the implications of an awareness campaign.

For anything that falls short of game changing, do not be afraid of culling these kind of features from your product backlog before anyone gets a chance to build them. If an idea truly is worth building, you won't be able to keep it from reappearing time and time again in the backlog.

Features where Vision ≠ Reality

What a user wants vs what they need. Features that fall into this category are those that are prescribed to be essential by users who do not yet have a use for them. This is sometimes due to ignorance, sometimes wishful thinking.

These features may present themselves as being Mission Critical due to received wisdom, especially in industries that have not recently been disrupted.

Suppose your company builds an advanced web analytics suite. You came from a background in Big Marketing where this was an essential feature. But your clients have ended up being mostly small businesses. Like you, the folk at the small businesses are thinking big; they dream of the game-changing insights you might help them uncover. But in reality, they have almost no data for you to work with and all they really need is a way of correlating banner impressions with online sales.

Features that fall into this category may also be the product of a vocal minority. If those taking part in your market research do not accurately represent your target audience, you may end up building 90% of your product for 10% of your users. Beware those who shout the loudest, and try to avoid launching into Build Mode as a knee-jerk reaction to every shiny new feature request.

Start with the obvious, then experiment

When you are trialling new ideas or deciding what to do with old ideas, objectivity is key.

Build the obvious Mission Critical features and spend the time needed to validate your Stroke of Genius ideas. Work in cycles and advance your bold ideas a bit at a time, doubling down on them when the data shows you're onto something, and cutting your losses when it's obvious you made a mistake.

Always avoid the temptation to build shiny and complex things where Vision ≠ Reality just because they excite you, and make a habit of frequently—and brutally—culling unvalidated Waste of Time ideas from your backlog.

As a business and as an individual, you only have so much time and money, so use both wisely. Keep it simple, triage features with a detached objectivity and avoid the obvious traps.

Why you shouldn't hire clones.

Adam Farrell — Fri, 23 Apr 2021 08:38:51 +0000

We have all been there. One of your best employees hands their notice in and your stomach does a little flip as you know how important they are.

You try your best to see if you can do anything to keep them but they explain this is an opportunity they can't turn down. So your attention turns to trying to find someone to replace them.

You tell your recruiter we need another "Sarah"! You want someone with the exact same skills so they can come in and "hit the ground running".

Is this the right approach?

Pause for a moment and ask yourself why would someone move roles to do the exact same job they are doing now?

Of course, this does happen sometimes, for example, if someone has been made redundant.

If you did hire that person how long will it take for them to get bored doing the same thing they have done for the last 3 years? Probably not long. This will likely mean after spending 1 year to 18 months with you they start looking for something more challenging and will provide better progression opportunities.

Then you are left in the same place as they were before, looking to hire someone doing the exact same job because you need them to "hit the ground running". And so the cycle continues.

Still, the biggest reason I hear for people looking to move roles is so they can learn and develop. If you can't provide that to a candidate then it will cause more pain further down the line.

So rather than going for a like for like replacement consider if hiring someone who perhaps doesn't have all the skills just yet but has the potential, ambition and correct attitude to attain them. If someone has something to work towards or aim for my experience is that they are far more motivated in the position.

Yes, this will mean a bit more time is needed on your part but that initial investment will usually lead to a brilliant new employee. I have seen it many times where people in this scenario have gone on to be a brilliant contributor to the business and they have more loyalty and appreciation for the business as they have profited as well.

So, don't just mindlessly look for clones of your previous employees.
Look at your current roles and expectations. Are they realistic? Hire for potential.

Load testing with Atata and Selenoid (Part 1)

danieltallentire — Thu, 15 Apr 2021 22:12:04 +0000

I'm Technical Director at Parker Software, a company with a long history of making market-leading live chat software and automation components.

I'm writing this article as part of CTO West Midlands writing group.

Atata is a great open source project created by @yevgeniyshunevych

What are we doing
Why Atata?
The Solution

What is load testing normally like for WhosOn?

Normally, we run our load tests in WhosOn using our APIs. This allows us to simulate the volume of requests coming in to the WhosOn server, but with a cheap and easy setup.
These are normally load tests like "1,000 concurrent chats" or "500 connected operators".

What is wrong with doing this?

We've recently come across some instances where load has caused a problem, but not to do with the overall capacity. The problem has been caused by behaviours of customers and operators that cause spikes. These spikes may cause an overload in a single component of the system, or a cascade through the system.
The spike behaviour might not be simple - it may have multiple factors.

If your load test is too simple it doesn't capture the real world behaviour of the operators.

What behaviour do we want to replicate?

We would like to create a load test that can simulate a shift change using our operator web client. During shift change, a backlog of queuing chats will build up - this could reach 100 chats in the queue, and as the new shift comes on board, logging in 50-100 users within a minute, those chats are then allocated out to the new operators. During this period, supervisors are also executing a variety of real time reports and pulling metrics against operators who are finishing up their shift, and monitoring the queue lengths to make sure that the active customers get serviced promptly.

Why Atata and not one of the regular load test systems like JMeter?

We already use Atata within our testing pipelines. Our devs are familiar with it.
It allows us to easily vary our load tests, and select tests for different scenarios by altering our coded tests.
It lets us execute tests in a variety of different browsers.
We can record screenshots or video the results if we want to see what the experience is like.

The solution

As part of our Azure DevOps process, we will create a test suite that can be executed for every release.
This will run a two stage process.

test the overall chat system with pick up by a bot.
test the client login system, with pick up by a UI automated chat agent.

We will use Selenoid to run a cluster of selenium instances in docker containers. This could be swapped out for any selenium cluster.

An XUnit test will drive the overall process, and a threshold will be set on various metrics to show whether that test was successful or not.

The test will also output a test file reporting on key metrics from the cluster, including how long the chats took to be handled, end-to-end message time and similar.

The code

I already started an earlier attempt at creating an Atata load test tool a few weeks ago. This was just using a basic set of browser on my box, but I ran into a lot of problems.

Firstly, it killed my computer, trying to pop open new browser windows, and second, it wouldn't be easy to transfer this into the DevOps pipeline.

Setting up Selenoid

I'm running on windows, so a few hoops to jump through to get Selenoid up and running:

update docker
change docker to use linux mode
force docker to use the correct linux context and permissions & 'C:\Program Files\Docker\Docker\DockerCli.exe' -SwitchDaemon
download the correct version of the selenoid configuration manager from https://github.com/aerokube/cm/releases
run ./cm.exe selenoid start to start up selenoid
run ./cm.exe selenoid-ui start to start up the UI so I can see what's happening

Now we have selenoid running, we can open up the selenoid UI on localhost:8080 and see what the connection details are for selenoid:

Building a global context

Based on this, and the Atata documentation, I created a global context driver implementation:

var capabilities = new DesiredCapabilities();
capabilities.SetCapability(CapabilityType.BrowserName, "chrome");
capabilities.SetCapability(CapabilityType.BrowserVersion, "89.0");
var driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), capabilities);

AtataContext.GlobalConfiguration
    .UseDriver(driver);

Creating a runner class

Since I need to run multiple Atata instances, I don't want to launch Atata as normal from a test - I need to create some tasks that can run Atata within them and maintain a separate context.

AtataContext.ModeOfCurrent = AtataContextModeOfCurrent.AsyncLocal;

This line of code instructs Atata to ensure that each asynchronous code block has its own instance of the Atata context.

Building the context

private void CreateContext()
{
    _context = null;
    var contextBuilder = AtataContext.Configure();
    _context = contextBuilder.Build();
}

public async Task SetupContext()
{
    await Task.Run(() =>
    {
        CreateContext();
        finishTime = DateTime.Now.AddMinutes(2);
    });
}

This creates the individual context for each LoadTestRunner

Running the test

public async Task Run(CancellationToken cancellationToken)
{
    AtataContext.Current = _context;
    try
    {
        var chatPage = Go.To<Survey>(url: "http://192.168.1.70/newchat/chat.aspx?domain=www.test.com")
            .VisitorName.SetRandom()
            .StartChat.ClickAndGo<Chatting>();

        while (!cancellationToken.IsCancellationRequested)
        {
            if (finishTime <= DateTime.Now)
            {
                chatPage.CloseWindowButton.Click();

                return;
            }

            await Task.Delay(10000);

            chatPage.ChatReply.SetRandom()
                .SendButton.Click();
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.ToString());
    }
}

This does a really simple start chat function, with only filling in the name field, sending a message every ten seconds, and closing after two minutes.

Time to try it out

I gave it a run, with a single session executing.
Success! It created a session in selenoid, and completed the test successfully - I got a chat popping up.

OK - time to up it to five sessions.

FAIL it only launched one session, and I got a ton of errors.

What went wrong?

Digging in to the debugging, I found that it is only launching one instance of the driver, not multiple as expected.

I changed the global code to:

var capabilities = new DesiredCapabilities();
capabilities.SetCapability(CapabilityType.BrowserName, "chrome");
capabilities.SetCapability(CapabilityType.BrowserVersion, "89.0");

AtataContext.GlobalConfiguration
    .UseDriver(() =>
    {
        return new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), capabilities);
    });

This now lets me launch 5 simultaneous sessions.

Can it go further?

I played around with parameters to reduce the memory and CPU usage, but the most I got working was 10 sessions - this partly seems to be to do with my IIS as well - something internally is preventing the sessions from connecting. Indeed, during the test my box prevented a manual chat session request from loading. My cpu was maxed out, and the memory usage was very high. This should work better in the cloud.

This is the selenoid setting I used to create a new instance with limited CPU and memory usage:
./cm.exe selenoid start --vnc --tmpfs 248m -g "-limit 20 -mem 128 -cpu 1.0"

What is next?

For Part 2, I'm going to explore moving this into our Azure DevOps Pipeline.

I need to optimise the memory usage further as well to ensure we can use more of these instances for a quick burst of testing.

Burnout Timeline

rich_marshall — Wed, 07 Apr 2021 19:32:01 +0000

Forgive me.. I have two threads of thought concurrently running through my head. They do join up but bear with me..

A little while back I started to write a post about feeling exhausted, tired, uninspired, possibly even burnt out. It was the day after I'd resigned. Over the previous months I'd become depressed, emotional and generally fed up. Work held no interest for me and I needed to rekindle my enjoyment of life.

The day I wrote the post I came to the conclusion that I was burnt-out. Work had been emotionally draining for months on end. The company was running out of money and it was becoming unclear if we'd even be able to pay the next month salary. At the same time I was expected to push the teams to deliver promises made to customers because it was our only revenue stream. On reflection, I don't think it was what is more widely recognised as burn-out but that day I felt a weight lifting from my shoulders and for the first time in months I genuinely felt happy....

Over the past few years I've been acting as a mentor for a very inspiring young gent who seems to have boundless energy. For the sake of this story, we'll call him Dave. Dave had got in touch with me through a bootcamp called School of Code who we both support. He was helping to teach the current cohort and I was helping with some longer term career guidance for the new recruits. When Dave got in touch he asked if I'd be his mentor. I was of course flattered but said at the outset I didn't know what I could do to help but was happy to try, if he was willing.

Our mentor/mentee arrangement seemed to work OK - I'd been given some instructions by someone I'd consider my mentor, though we'd never entered into a formal arrangement, on how we should engage. Above all else, she told me that a mentee should be trying to achieve something and a mentor is there to help guide them to their goal.

Having agreed to a clear objective with Dave and how I could help reach that, we agreed to meet, or at least talk every few months. A few meetings in we were talking about how he was starting to feel worn out, but more concerning how he was doubting himself. I asked what he was doing that was leading to this feeling. Apart from his day job as a developer, he was also working on a side project, and also teaching a day each week at School of Code. He was also keen to shift into a more product oriented role. Wow... So I was just trying to the one full time job - Dave was determined to do 3!

At this point I asked him why he was putting so much pressure on himself "because I want to achieve my objective!". I pointed out how much he was already doing and that struck me as being pretty successful already.

Then I asked him;
"How old are you?"

22 he said :O

And then followed something which I'd paid more attention to myself.

"You're 22. Look at your career as a timeline. You started at 19, 20? Perhaps you hope to retire at 50. maybe 60. many people will need to work until they're 75. Now, look how far you are along your timeline. You've barely started your career! Give yourself a break! I didn't even start to think about my career ambition until I was into my 30s!

Basically, relax, give yourself a break - you're already doing brilliantly. Pace yourself. You need to enjoy your job and your life."

I can't promise this was was exactly what I said, but I know it was close. At the time I said it, it seemed to click with Dave who gave a sigh of relief.

We carried on as mentor and mentee but ended up talking less over the past year. When we did speak again, just recently he mentioned to me that he'd been using that "timeline advice" with some of the people he'd been teaching and mentoring at School of Code, among other places. I'd totally forgotten about it so asked to be reminded. He told me the number of people he'd used it with and how all of them had appreciated the context it gave them and the assurance that they didn't need to break themselves to achieve what it is they hoped for.

I was chuffed to bits to hear that something I'd come up with on the spot was helping him and others and made a note to make more use of it myself in future..

This last conversation happened a few days after I resigned and felt the weight lifting from my shoulders. A few days after that I reflected; why didn't I take my own advice, dammit!?

I came to realise that I'd stuck out the job I was no longer enjoying because I saw it as a stepping-stone or launchpad. Where else was I going to be a CTO at a FinTech, supposedly a prestigious industry? What else was going to help progress my career to the next step? I'd got myself stuck in that "career progression" mindset and I'd totally lost sight of enjoying what I did and importantly my life.

If I'd taken a moment to look at my own career timeline, perhaps I would have seen that I'm also doing OK. That I don't need to get stressed and more importantly, I need to do something I'm going to enjoy.

So you see - the threads do join up, and the result of that is I need to listen to my own advice more.

The other thing I've learned is that there's always a new opportunity out there. I spend most of my professional life encouraging others to embrace change (and get used to the idea that deploying code 20 times a day is both normal and a good thing!). Sometimes I need to believe that myself and believe in myself.

Using Atata to create UI tests for user defined web forms

danieltallentire — Fri, 26 Mar 2021 08:58:25 +0000

The Problem

At Parker Software, our chat software allows for customers to create whatever inputs they want to show on the pre-chat survey, which is asked before a user enters the chat:

These fields can be any html field type, including simple text inputs, password fields, radio buttons and drop down lists.

When we setup a new theme for a customer, we want to ensure that the layout and behaviour remains the same when any updates and modifications to the core chat window take place. At the moment this is a manual process whereby the upgrade is performed, and a QA team member will run through the chat window, inspecting it from errors. This manual process is somewhat error prone.

I want to create an automated solution that can work regardless of the fields that have been set in the drop down

We use Atata to do some of our other automated UI testing, and I wanted to have a go at using it to solve this problem.

What is Atata?

Atata is an automated testing framework based on Selenium. You can use C# code to define pages as class files, then really simply set up a wide variety of tests without needing to worry about what the client side looks like.

Thanks to Yevgeniy Shunevych for making Atata so awesome and putting out continuous updates and improvements.

What does it look like?

What are the challenges?

Atata doesn't have a "dynamic" control built in - it usually requires all elements that you want to interact with to be predefined, so I will need to spend some time working on resolving that.

The setup

I'm using Atata with XUnit - we use XUnit for all our unit testing, so Atata has to run through that - it is a little more difficult as Atata doesn't have a native XUnit adapter, but only a few extra changes are needed to integrate with XUnit's logs.

The code

You can get the code at https://github.com/danieltallentire/AtataDynamicForms

Attempt 1

First, I attempted to use the ControlList class with the built in Input control.

using Atata; 

namespace AtataDynamicFormTester 
{ 
    using _ = ChatStartPage; 

    [WaitForLoadingIndicator] 
    class ChatStartPage : Page<_> 
    { 
        [ControlDefinition(ContainingClass = "input-group")] 
        public ControlList<Input<string, _>, _> SurveyFields { get; private set; } 

        public Button<Chatting, _> StartChat { get; private set; } 

        public _ SetRandom() 
        { 
            foreach (var field in SurveyFields) 
            { 
                field.SetRandom(); 
            } 

            return this; 
        } 
    } 
}

Each control on the form is wrapped in a div with the input-group class. This means we can use it with the list locator to find each of the inputs inside.

This came unstuck pretty quickly. It was able to detect and find the elements in the control list, but unable to set a value on them, even with a simple text field.

Attempt 2

My next attempt was to use a simple custom control to wrap the input control. In Atata a custom control is a class derived from the parent Control.

The containing class now moves to part of the control definition.
We can rely on this element existing, so we can iterate the fields.

    [ControlDefinition(ContainingClass = "input-group")] 
    class DynamicControl<TOwner> : Control<TOwner> where TOwner : PageObject<TOwner> 
    { 
        [FindByClass("form-control")] 
        public Input<string, TOwner> TextInput { get; set; } 

        public void SetRandom() 
        { 
            TextInput.SetRandom(); 
        } 
    }

The list is then changed to reference it:

  [ControlDefinition(ContainingClass = "input-group")] 
        public ControlList<DynamicControl<_>, _> SurveyFields { get; private set; }

Success!

Well... some success. This worked fine for simple fields like Name and Company. It got unstuck when it reached an email address, trying to fill it as a random field, and completely ignored the other controls.

My next thing to change was to try to switch based on the input value:

public void SetRandom() 
{ 
    switch (Input.Attributes.Type) 
    { 
        case "text": 
            Input.SetRandom(); 
            break; 
        case "email": 
            Input.Set(Atata.Randomizer.GetString("{0}@{0}.com")); 
            break; 
        case "date": 
            Input.Set("03/03/2021"); 
            break; 
        case "checkbox": 
            Input.Set("true"); 
            break; 
        default: 
            // ignore 
            break; 
    } 
}

OK. Getting closer - this worked nicely for the email address and the date - although its a bit clunky setting the date as a string.
Checkbox didn't work at all, and the select wasn't even detecting.

Attempt 3

Looking again at the classes, I'd forgotten that select elements aren't input elements... d'oh!

I had a play around, and found that I could add multiple elements in to the class:

        [FindByClass("form-control")] 
        public Input<string, TOwner> Input { get; set; } 

        [FindById("fld")] 
        [TermFindSettings(TargetAttributeType = typeof(FindByIdAttribute), Match = TermMatch.StartsWith)] 
        public CheckBox<TOwner> Checkbox { get; set; } 

        [FindByClass("form-control")] 
        public Select<TOwner> Select { get; set; }

And it wouldn't error out for the missing ones in the cases where the control didn't contain those elements, however if I tried to access the control Input on a select element it would throw an exception (rightly, because the Input doesn't exist).

I found that I could create a check to see if the element was found by checking for the Exists property with the IsSafely flag set:

 if (Checkbox.Exists(new SearchOptions() { IsSafely = true }))

This caused a slowdown, so this lead me to the final code with a predefined timeout:

 [ControlDefinition(ContainingClass = "form-group")]
    class DynamicControl<TOwner> : Control<TOwner> where TOwner : PageObject<TOwner>
    {
        [FindByClass("form-control", Timeout = 0.01)]
        public Input<string, TOwner> Input { get; set; }

        // checkboxes aren't wrapped by a nice form-control wrapper, so have to use a different search method
        [FindById("fld", Timeout = 0.01)]
        [TermFindSettings(TargetAttributeType = typeof(FindByIdAttribute), Match = TermMatch.StartsWith)]
        public CheckBox<TOwner> Checkbox { get; set; }

        [FindByXPath(".//*[contains(concat(' ', normalize-space(@class), ' '), ' form-control ')]/descendant-or-self::input[@type='date']", Timeout = 0.01)]
        public DateInput<TOwner> DateInput { get; set; }

        [FindByClass("form-control", Timeout = 0.01)]
        public Select<TOwner> Select { get; set; }

        public void SetRandom()
        {
            if (Checkbox.Exists(new SearchOptions() { IsSafely = true }))
            {
                Checkbox.Set(true);
            }
            else if(Select.Exists(new SearchOptions() { IsSafely = true }))
            {
                Select.Set(Select.Options[Atata.Randomizer.GetInt(0, Select.Options.Count - 1)].Value);
            }
            else if (DateInput.Exists(new SearchOptions() { IsSafely = true}))
            {
                DateInput.SetRandom();
            }
            else if (Input.Exists(new SearchOptions() { IsSafely = true }))
            {
                switch (Input.Attributes.Type)
                {
                    case "text":
                        Input.SetRandom();
                        break;
                    case "email":
                        Input.Set(Atata.Randomizer.GetString("{0}@{0}.com"));
                        break;
                    default:
                        // ignore
                        break;

                }
            }           
        }
    }

I needed to use a manual XPath search for the date input - this was because Atata by default allows for date inputs to be a normal text input with a mask, so was also picking up the default text boxes as date fields too.

I had to add a new randomizer to make the DateInput SetRandom work.
Now I've got the chat window starting nicely.

The containing class was actually incorrect in my earlier examples, and I changed it to form-group - this was caused by the checkboxes not being inside an input-group div.

Just enabling screenshotting in the places I want them:

        public void FillPrechatSurvey()
        {
            Go.To<ChatStartPage>(url: "http://localhost/newchat/chat.aspx?domain=www.parkersoftware.com")
                .Report.Screenshot("Start Chat")
                .SetRandom()
                .Report.Screenshot("Filled Out")
                .StartChat.ClickAndGo()
                .Report.Screenshot("Chatting")
                .Wait(5)
                .CloseWindowButton.Click()
                .Report.Screenshot("Finished");
        }

And the results:

What next?

In order to make this good for production I've got to do some little tweaks - making the resolution and the URL something that can be passed in as an environment variable so it can be run inside DevOps. This is reasonably straight forward.

In the longer term, I want to be able to configure some built in routes and selections to validate that the data sent in the set random is what is received by the server. The WhosOn Echo Bot can send the data back, but I'll have to save the output somehow with SetRandom.

Update

Yevgeniy suggested some refactorings to make my rough code a little neater:

 [ControlDefinition(ContainingClass = "form-group")]
    [FindFirst(TargetAllChildren = true)]
    class DynamicControl<TOwner> : Control<TOwner> where TOwner : PageObject<TOwner>
    {
        // checkboxes aren't wrapped by a nice form-control wrapper, so have to use a different search method
        [FindById(TermMatch.StartsWith, "fld")]
        public CheckBox<TOwner> Checkbox { get; set; }

        public DateInput<TOwner> DateInput { get; set; }

        public EmailInput<TOwner> EmailInput { get; set; }

        public Select<TOwner> Select { get; set; }

        public TextInput<TOwner> TextInput { get; set; }

        public void SetRandom()
        {
            if (Checkbox.IsPresent)
            {
                Checkbox.Set(true);
            }
            else if(Select.IsPresent)
            {
                Select.Set(Select.Options[Atata.Randomizer.GetInt(0, Select.Options.Count - 1)].Value);
            }
            else if (EmailInput.IsPresent)
            {
                EmailInput.Set(Randomizer.GetString("{0}@{0}.com"));
            }
            else if (TextInput.IsPresent)
            {
                TextInput.SetRandom();
            }
            else if (DateInput.IsPresent)
            {
                DateInput.SetRandom();
            }

        }
    }

Changing the inputs to use IsPresent is much more succinct.
Adding the top level attribute for TargetAllChildren means that the class name isn't required.

Using TextInput instead of date input means that the order can be changed to allow a better ordering, and get the correct text or date input.

For this to work I had to add an EmailInput as well to catch type='email'

'Just because I can build this, should I?'

David Maidment — Thu, 18 Mar 2021 08:49:20 +0000

When it first became available to anyone with a telephone line and a modem, The World Wide Web was a very special place. Every trip onto the Web was a voyage of discovery. Content was being created at breakneck speeds, and behind every website was someone who took on the mantle of 'webmaster' for the pure love of the subject matter.

Interested in the ecology of Iceland? The works of Chaucer? The Powerpuff Girls? You could guarantee there would be at least one website on the subject painstakingly curated by someone just as fanatical as you. Someone who cared so much about the subject they taught themselves HTML just to share what they knew. It was fandom-driven-development.

The Web was also a beautiful maze.

There were no search engines. No algorithms telling you what to look at next. Nothing to serve up content specifically to support your worldview. Anything you wanted to see, you had to go out and specifically look for it. And if in the process you came across a little pocket of culture that was foreign to you, you would learn something new about the world. There were no comments sections telling you that what you had just read was wrong and that the author should harm themselves—you were trusted to use common sense to form an original opinion.

Perhaps most importantly, The Web was a place where communities sprang up organically.

In 2021, companies pour huge amounts of time and money into curating communities—Chief Community Officer is a real job title you can hold. It's taken a long time, but the importance of communities is increasingly being recognised by businesses.

But back in the 1990s? Any enthusiast with a modem was capable of bringing together hundreds or thousands of strangers. Those online communities came about naturally, and from them, you could make friends. Real friends. Friends you would stay up until the early hours of the morning talking to about everything and nothing. Far-flung friends who would teach you about their interests, their part of the world and their culture. You never stopped learning.

In those days I ran a number of vibrant communities, none of which were intentional. They just happened, because humans are good at organising into groups. It's literally in our DNA. That evolutionary trick helped us grow from cavepeople to a global civilisation. It put people on The Moon. But working in groups requires trust, and trust is built by getting to know someone.

We tend now to treat the problem of 'community' as though it's a new one, but it isn't. We've always known how to build them. Decades ago, online, they were everywhere. So what changed?

In the early 2000s, The Web took a hard right as Social Media took off. The importance of The Individual over The Community soared, and many of the old vibrant communities migrated to social platforms to die a slow, undignified death.

Why did this happen?

Once advertising became a viable online business model, finding ways to make users return to your site every day took centre stage. Almost overnight, dopamine became the currency of The Web, and interactions became more individual-centric. Where we used to have to get to know someone and form a real, human connection, it became the norm to broadcast short messages about ourselves—Status Updates, which we all knew and loved as the taglines of our IM profiles, now without the need to have the accompanying conversation. And when typing became too much work, Likes simplified the process even further. We collected hundreds of 'friends', knowing precisely nothing about most of them. It was how all the platforms operated, so it became the new normal.

To keep the dopamine flowing, the new generation of community tech worked hard to avoid negative experiences. Keeping you in an echo chamber is the easiest way to do this: see more of what you like, and less of what you do not. On paper it sounds like a good idea, but in practice it deepens trivial political divides and removes healthy challenges to your worldview.

Soon enough lines were being drawn everywhere between imagined us and thems. We forgot that life is messy, people are not perfect and that there is nuance is every single situation; that nothing is simply good or evil; that some things require considered conversations rather than emojis. We stopped learning about each other and started making dangerous assumptions about each other.

It. Was. Devastating. We had lost something truly beautiful.

Growing up with technology, I was in constant awe of the possibilities. With The Web, I saw those possibilities flourish as communities self-organised around fansites, encouraging endless learning, conversations and friendships. The future was going to be something extraordinary, and it was all thanks to technology.

But here's the thing: while technology can be amazing, it is not amazing in and of itself; it requires us to make it amazing.

We make technology. Technology that can go viral overnight and gain a large percentage of the world's population as users. So we have a responsibility to ensure that the things we create are carefully considered at every stage. No one set out to create today's individualistic online culture, and that's the most terrifying part—that it happened not because someone said, 'Let's change online behaviour for the worse,' but because they had to figure out how to monetise platforms that people were increasingly expecting to access for free.

We have a responsibility to ensure that the technology that powers the next twenty years is better than the technology that powered the last twenty years.

Regaining the lost potential of The Web (which I honestly believe is in self-organising, tangentially educational, social groups) is not a complex problem, but it will take a concerted effort to solve. If enough technologists work to reframe what is 'normal' online, we can undo some of the damage accidentally wrought by The Web. And it starts by asking a simple question at the beginning of each new project: 'Just because I can build this, should I?'

First 90 days as a new CTO

rich_marshall — Fri, 12 Mar 2021 09:08:33 +0000

A few weeks ago there was a question posed in a slack group: "If you could go back to your first day in the role what are the 3 pieces of advice you would tell yourself?"

There were some really helpful responses in there: "Read a lot of books and speak to other CTOs as early as possible"; "Spend more time thinking and less time doing in the first 90 days"; "Believe in yourself - there’s a reason why they asked YOU to be CTO" - that last one was my suggestion, but as an evergreen sufferer of imposter syndrome, I have to keep reminding myself.

The question got me thinking though. Pithy advice is certainly helpful, but where do you go from there? So beyond the advice, what's my plan to hit the ground running in my next leadership role? This post is partly me sharing my thoughts but is also a request for peer review - I'd love to hear back on what I've missed, what I'm over emphasising - so please feel free to post back and comment. I should share my assumptions here too - the team is somewhere between 10 and 50 people and a new leader is being hired because the company is growing or because they want to trigger change.

Starting a new role is an opportunity to start with a clean sheet, to take all the things you've learned to date and and try and apply them in what feels like a more sensible approach and order. So here's my 90 day plan - what's missing, what's un-necessary? Where am I setting a trap for myself?

Day 0 - introduce myself
- Tell the team about who I am, what I believe in and how I like to work. Give the team a chance to mentally prepare themselves for my arrival. Importantly share my intentions with them.
Day 1 - (organise to) meet the team, listen. Hear what they have to say. What do they love, what bugs them? What insight can I gain from them?
- Meet with all of them and make notes - individual notes for each person.
- Make sure I have time after each person to write up my notes and digest what they've said
Week 1 - Establish my place in the company
- Establish what my peer leadership group needs from me, what expectations do they have - "role descriptions' are rarely well defined at this point.
- Form an engineering leadership group, if there isn't one - this will likely be my go-to team as I learn the ropes.
- Start to lay-out the common working patterns
- Describe the culture that I'm looking to build
- Start to establish team principles and purpose
- Measure, or identify the tools that will allow us to measure
- Build trust with the team and show them how I plan to work with them in future
Month 1 - Establish frameworks for how I want to support and grow people
- Agree principles. Share them, adopt them, refere to them at every point
- Share any feedback or insights already learned
- Start to create framework for professional and personal development - there are some great examples out there for re-use. E.g. Basecamp and Gitlab company handbooks.
- Check-in with peer leaders. How am I shaping up? What am I missing?
Quarter 1 - "Standardise" on measurements, learning and feedback
- Socialising what we're learning from measurements, use this to establish what's working well and what needs improving
- Forming tactical and strategic plans
- Team check-ins - how am I doing?

Taking those tips into account too, I don't plan to change things in a hurry, unless I really need to - the first 90 days are about learning and listening.

I've mentioned measurement in there a few times. I'm really interested in knowing how the teams are performing, not so I can make them work harder, but so we can learn to work smarter. I think Accelerate is a fantastic book that goes into the science of high performing teams. Measuring how we're performing as a team is vital in being able to learn and improve. Likewise I'll be looking to resolve specific pain points. Over the past 18 months, Team Topologies has become my "go-to" when trying to understand what's not working well and how we can think about improving it. It's a tool box of patterns to describe and promote people working well together within teams, and how different teams should work together and importantly know how and when not to work together. I also believe strongly in teams being given purpose and empowered to achieve their goals. Distilling company values into more readily consumable principles is an important part of making this possible while ensuring there is still consistency across those teams. I've found Simon Sinek, especially in "The infinite game" and "start with the why" brings purpose and clarity to my thinking and provides a valuable anchor when I need to take a step back and reset.

Finally, there was another piece of advice which was given, and I still can't decide if I totally agree: "your team is the exec team now, not the development team". If there's one thing I've learned in my current role is that if the exec don't act as a team, then you're not going to be effective as a business, so I agree the exec team is absolutely your team.. and yet I can't help think that the tech team is also my team. If I have the vision, they're the people that make it possible. If I have a problem, they're the people that are likely to solve it. If we're not in it together, working towards out purposes then we're all just "me" or "them" and that's not the path to success either. For me it's got to be "us" all the way, so thank you but I'm in both teams now :)

DEV Community: CTO UK - West Midlands

Why product development is the beautiful game

'I wonder if...': Moving ASCII art

Make it so

Playing it all back

Improvements I should have made

Should you build it, or abandon it?

Mission Critical features

Stroke of Genius features

Waste of Time features

Features where Vision ≠ Reality

Start with the obvious, then experiment

Why you shouldn't hire clones.

Load testing with Atata and Selenoid (Part 1)

Contents

What is load testing normally like for WhosOn?

What is wrong with doing this?

What behaviour do we want to replicate?

Why Atata and not one of the regular load test systems like JMeter?

The solution

The code

Setting up Selenoid

Building a global context

Creating a runner class

Building the context

Running the test

Time to try it out

What went wrong?

Can it go further?

What is next?

Burnout Timeline

Using Atata to create UI tests for user defined web forms

The Problem

What is Atata?

What does it look like?

What are the challenges?

The setup

The code

Attempt 1

Attempt 2

Attempt 3

What next?

Update

'Just because I can build this, should I?'

First 90 days as a new CTO