DEV Community: Jaakko Kangasharju

I had a bug with ForEach and Range in my SwiftUI code

Jaakko Kangasharju — Wed, 29 Apr 2020 18:56:29 +0000

I have a side project for iOS, and since it's a side project, I thought to try SwiftUI for the UI. There's some complexity because the UI isn't exactly traditional, but overall, I've been happy with SwiftUI.

But when Xcode 11.4 was released, the app wouldn't work properly. Some, but not all, parts of the UI weren't updating anymore when state changed. For a while, I stayed on 11.3 so I could work on the functionality, but as time passed, it became evident that this wasn't a problem with Xcode or iOS, it was something in my code.

Mostly, I thought that I had messed up with the bindings so that the UI wasn't observing my presentation layer properly. No luck there, nothing I tried would work. After a lot of frantic Web searching, I eventually found the answer in the comments to an unrelated blog post.

My code looked something like this

struct MainView: View {
  @Binding var names: [[String?]]

  var body: some View {
    VStack {
      ForEach(0..<self.names.count) { nameIndex in
        NamesView(self.$names[nameIndex])
      }
    }
  }
}

The problem is that ForEach has three init methods,

init(Data, content: (Data.Element) -> Content)
init(Range<Int>, content: (Int) -> Content)
init(Data, id: KeyPath<Data.Element, ID>, content: (Data.Element) -> Content)

and using it like I did picks the second one, with the Range parameter. This method assumes that the range doesn't change, which is true in my case, but it also seems to assume that the content views don't change, at least from Xcode 11.4 onward. Maybe this is a bug, maybe it's intended behavior, I can't tell from the documentation.

When I had finally discovered the problem, the fix was simple enough. I changed the ForEach to

      ForEach(0..<self.names.count, id: \.self) { nameIndex in
        NamesView(self.$names[nameIndex])
      }

which picks the third init method of ForEach, and allows updates of the views.

Detecting Cute Animals on iOS: A Continuation

Jaakko Kangasharju — Mon, 21 Oct 2019 07:43:19 +0000

Earlier, I wrote about making a cute animal detector with machine learning for a mobile phone. Back then, I had to develop only for Android, since the iOS tools were incompatible with what I was using to train my model. But now the iOS tools are updated, and I had some time, so here is what I did to implement my cute animal detector on iOS.

The earlier article explains all the model training that needs to be done. Here, I talk about only the additions that needed to be made to make the app for iOS. The repository has been updated with the latest code.

Converting the Model

On iOS, CoreML is the format to use for machine learning models. Apple provides tools that are able to convert the Keras model I have into CoreML in a straightforward manner.

import coremltools

coreml_model = coremltools.converters.keras.convert('iscute.h5',
    image_input_names='input1',
    input_name_shape_dict={'input1': [None, 299, 299, 3]},
    class_labels=['cute', 'notcute'])
coreml_model.save('iscute.mlmodel')

As in the Android case, the converter requires the input shape to be provided. In this case, however, the first dimension in the input tensor had to be left to None, not set to 1. The image_input_names and class_labels are needed so that the iOS app knows that the model 1) processes images, and 2) is a classifier. This makes writing the app code easier, as special-purpose APIs are available.

Reading the Camera on iOS

The app needs to do the same things on iOS as on Android:

Show the camera preview on screen
Let the user press a button to capture the current image
Feed the captured image to the classifier
Display the result after the classification is complete

The Android implementation of image capture did not actually use the camera. Instead, when the user pressed the button, it grabbed the preview contents as a bitmap and generated the image to classify from that. On iOS I did not find a way to extract the preview contents as a bitmap, so I went with the route of instructing the camera to take a proper picture.

Photo by Amy Chen on Unsplash

Before the app can access the camera, the user must grant it permission. There are other possible authorization statuses, and the user might deny permission when asked, but for a sample app that is only about using the camera, I consider it fine to assume that the permission is or will be granted.

switch AVCaptureDevice.authorizationStatus(for: .video) {
  case .authorized:
    self.setupCaptureSession()
  case .notDetermined:
    AVCaptureDevice.requestAccess(for: .video) { granted in
      if granted {
        self.setupCaptureSession()
      }
    }
  default:
    break
}

For custom use of the camera, as in this app, AVKit is the platform library to use. A camera session needs to be created and started. In a session, there are inputs and outputs, with inputs connecting to outputs. For this case, the input is set to video input from the back camera, so that the screen can show a live preview, and the output is a photo, since the app processes a single photo.

let captureSession = AVCaptureSession()
captureSession.beginConfiguration()
let backCamera = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back)
guard let backCameraInput = try? AVCaptureDeviceInput(device: backCamera!),
  captureSession.canAddInput(backCameraInput) else {
    return
}
captureSession.addInput(backCameraInput)
self.cameraOutput = AVCapturePhotoOutput()
guard captureSession.canAddOutput(self.cameraOutput!) else {
  return
}
captureSession.addOutput(self.cameraOutput!)
self.previewView!.previewLayer.session = captureSession
self.previewView!.previewLayer.connection?.videoOrientation = .portrait
captureSession.commitConfiguration()
captureSession.startRunning()

The photo output is what is used to take the picture. The target method of the evaluate button creates a settings object, where the default settings are fine for this use, and instructs the photo to be captured.

@objc func evaluate() {
  let pictureSettings = AVCapturePhotoSettings()
  self.cameraOutput!.capturePhoto(with: pictureSettings, delegate: self)
}

Cropping the Photo

In the Android app the app was processing a bitmap, which made cropping and scaling easy. On iOS, AVKit will produce an AVCapturePhoto object, which needs a bit more work to match with the screen, since the screen resolution is different from the photo resolution.

func cropImage(photo: AVCapturePhoto) -> UIImage {
  let photoOrientation = CGImagePropertyOrientation(rawValue: photo.metadata[String(kCGImagePropertyOrientation)] as! UInt32)!
  let fullCgImage = photo.cgImageRepresentation()?.takeUnretainedValue()
  let fullCgImageSize = CGSize(width: fullCgImage!.width, height: fullCgImage!.height)
  let bounds = self.view.bounds
  let orientedBounds = size(from: bounds.size, in: photoOrientation)
  let scale = fullCgImageSize.height / orientedBounds.height
  let frameSize = CGSize(width: 299.0 * scale, height: 299.0 * scale)
  let center = CGPoint(x: fullCgImage!.width / 2, y: fullCgImage!.height / 2)
  let frame = CGRect(x: center.x - frameSize.width / 2, y: center.y - frameSize.height / 2, width: frameSize.width, height: frameSize.height)
  let croppedCgImage = fullCgImage!.cropping(to: frame)
  let imageOrienation = imageOrientation(from: photoOrientation)
  let croppedImage = UIImage(cgImage: croppedCgImage!, scale: UIScreen.main.scale, orientation: imageOrienation)
  return croppedImage
}

The best image type in this case that supports cropping is CGImage. Unlike UIImage that would be used for on-screen display, a CGImage does not know its orientation, so that needs to be extracted from the AVCapturePhoto object. The orientation is used to make sure that the screen bounds are oriented in the same way as the photo, so that the scaling factor can be correctly computed. (The width and height scales were different; using height worked correctly but I’m not sure why.)

The picture to extract for classification is a 299x299 area in the center of the photo. So the code scales a 299x299 rectangle to the photo resolution, centers it, and does the cropping. Finally, a UIImage with the appropriate screen scaling factor and orientation is returned, since working with a UIImage is handier than with a CGImage.

Photo by Liliya Lisa on Unsplash

Running the Classifier

Creating the model is best done at startup since it is not going to change while the app is running, and it takes a while to create. The created model type is from the Vision framework, which is a nice library to use for image classification purposes.

required init?(coder: NSCoder) {
  let modelConfiguration = MLModelConfiguration()
  modelConfiguration.computeUnits = .cpuAndGPU
  do {
    let isCuteModel = try iscute(configuration: modelConfiguration)
    model = try VNCoreMLModel(for: isCuteModel.model)
  } catch {
    return nil
  }
  super.init(coder: coder)
}

When trying to run the model with the default configuration, I encountered an Overflow Error that apparently means the model is too big for the neural engine. So that is why it is limited to CPU and GPU only.

Starting the classification with the Vision framework is straightforward. Create a classification request with the model and a callback method, create a handler from the (cropped) image, and call the handler with the request.

let image = cropImage(photo: photo, photoOrientation: photoOrientation)
let request = VNCoreMLRequest(model: model, completionHandler: pictureClassified)
let handler = VNImageRequestHandler(cgImage: image.cgImage!, orientation: photoOrientation, options: [:])
try handler.perform([request])

When the classifier has finished, the completion method will be called with the results. The method verifies that the results are classification results. Since the model was converted as an image classifier with the appropriate labels for the outputs, the code can look for the identifier cute instead of needing to know which index corresponds to which class.

func pictureClassified(request: VNRequest, error: Error?) {
  guard let results = request.results as? [VNClassificationObservation] else {
    print("Fail \(String(describing: error))")
    return
  }
  for classification in results {
    if classification.identifier == "cute" {
      showResult(classification.confidence)          
    }
  }
}

Summary

Building the iOS app was somewhat less work than building the Android app, in part because I already had some experience from the Android app. The biggest hurdle was the image processing that needed to be done on the captured photo, since I was not that familiar with the intricacies of UIImage, CGImage, and AVCapturePhoto. As after making the Android app, I am now more confident in my ability to apply machine learning in a real iOS app.

Screenshot of the app evaluating a dog

Yep, it still works.

Programming with State: Concurrency

Jaakko Kangasharju — Tue, 27 Aug 2019 10:58:11 +0000

Programming with mutable state is already difficult in a sequential execution model where only one part of the program is executing at one time and functions complete execution without being interrupted. The context of our applications is not so accommodating, though, requiring programs to respond to various asynchronous events. And while some languages, like Javascript, adopt only asynchrony, many languages bring in also concurrency, where many parts of the program might be (conceptually at least) executing at the same time.

Threads are a common abstraction for concurrency. Since each thread has access to the same program memory, the problems of mutable state get exacerbated. Not only can state now change between function calls, it can be changed by another thread while a function is executing. This problem is typically handled by using locks to prevent multiple threads from operating on the same piece of state at the same time, but locks can be difficult to get right.

Invariants

Looking at the value-entity model from the first post of this series, there are additional problems with multithreading. I mentioned invariants as a useful tool to keep the state of an entity manageable. But since an invariant can be temporarily violated while an entity is modifying itself, a concurrent execution of a state query might observe the inconsistent state. And this leads back to all the problems with mutable state that we were trying to avoid by adopting the value-entity model.

Locks internal to an entity can of course be used to ensure that other threads do not see temporary invariant violations. An additional problem to the usual ones in the entity model is that often we want to allow concurrent invocations on an entity that do not conflict with each other, and designing locking to support this can get complicated. Furthermore, a lock-based system only supports those operations that were thought of by the entity implementor. For instance, a common operation needed for an associative array is insert-if-not-present, but if the array only implements operations for checking the presence and unconditional insertion, this combined operation cannot be safely performed by the caller.

Software Transactional Memory (STM) is an interesting concept that solves many of the problems in lock-based entity-style programming. The optimistic execution makes non-conflicting operations succeed. The ability to wrap atomic operations into larger atomic operations solves the composability of operations. And finally, when programming with the observable-state model that emits modification events, STM prevents such an event from being emitted before all state modification has completed. STM is not very mainstream, though, so even though libraries exist for many programming languages, it would be difficult to justify its adoption.

Entities and Messages

From the word “entity”, one might expect a different kind of execution. An entity brings to mind something autonomous and separate, but this is not how threaded execution sees things. There are no enforced entity boundaries. Any thread can just come and execute an entity’s code at any time. What was architecturally good encapsulation in a single-threaded system completely disappears in a multi-threaded system.

My preferred system following this thinking would have each entity “living” in a single thread. All of the code belonging to an entity would always be executed on that entity’s thread. This is similar to many UI frameworks that require UI component manipulation to happen in the UI thread, except that I would prefer the system to take care of thread switching instead of having to do it manually. Such a system would make programming much more asynchronous, since each entity invocation could involve a thread switch, but I have written code in this style, and it is not too difficult with, say, the async/await mechanism.

This kind of view of objects as independent entities that communicate with each other is not new. It resembles Alan Kay’s definition of object-oriented programming, which is focused on messages between objects. When looking from the concurrency perspective, processes in Erlang function semantically as independent entities with internal state. To me, it is more difficult to conceptualize observable entity state in the message passing model, especially the dependency network of derived state as described in the first post. Still, when concurrency gets involved, I find seeing entity invocation as message passing a better way of comprehending the system in a way that helps make it work correctly.

Mitigation in Practice

There are some patterns and ways of thinking that can mitigate the problems of concurrency with mutable state. Unlike in the single-threaded case, these tend to be more difficult to set up so that they would work automatically. Therefore more discipline and careful documentation of the used patterns is needed, since the programming environment cannot always enforce it for us.

One way to avoid the issues with other threads observing temporarily-broken invariants is to postpone internal state modification. Instead of directly modifying an entity’s state during function execution, use local variables to capture all the modified pieces of state, and only at the end of execution set the entity’s state. This also helps when an invocation depends on operations that can fail, since no actual modifications have yet been made so the execution can simply be abandoned instead of having to implement complex rollback logic.

This style of state update becomes interesting when the state to update is some form of collection, like an associative array. Fully copying a big structure for local modifications may be too inefficient (but it might not; remember to measure before assuming where your bottlenecks are). A good option would be to use a persistent data structure, which allows the modifications to be made locally in a natural way, swapping the original for the updated one at the end, but persistent data structures are not commonly used in many programming languages. The ultimate expression of this style would be to keep all (mutable) state of an entity in a single persistent data structure, which would eliminate any issues with breaking invariants, since the state would then only get modified all at once in a single operation.

When an entity gets more complex, it can be helpful to structure its internals as a state machine, with the state and allowed transitions defining which invocations are permissible. This can also help in concurrent and asynchronous situations. Namely, when the state machine implementation is thread-safe, the check on permitted state transitions can take care that conflicting operations are not executed, even when requested concurrently.

Surprisingly, I’ve found that automated tests can be helpful. In general, concurrency issues are non-deterministic, depending on specific timing conditions that won’t be reproduced exactly. What I do, when applicable, is to run a huge number of threads in the automated test, all hammering on the entity that is supposed to be thread-safe, and verify that the results are consistent with some correct execution. This is not always possible to do, usually the expected result is not as clear-cut as with a traditional test, and the test may not always catch failures. But if the expectations can be defined, I’ve found that running enough threads flushes out many problems reliably enough.

Summary

Dealing with mutable state in a concurrent program can get very difficult. Concurrency makes it ever more important to architect the program so that state mutation happens only in specific, controlled ways. Building the program directly on low-level concurrency abstractions, as often provided by popular programming platforms, will usually lead to trouble. Whenever possible, try to extract higher-level concepts into reusable components, such as task executors or operation queues. Be even stricter in controlling how state is mutated. And when it is not possible to ensure correct operation through language-enforced mechanisms, be very disciplined and document carefully what is being done and why.

Programming with State: Architectural Roles

Jaakko Kangasharju — Tue, 20 Aug 2019 12:06:06 +0000

To me, application architecture has one primary question to answer: Where does this functionality belong? An application I’m working on is going to have features, each feature consisting of several pieces of interacting functionality. And I want to have a clear view of where each piece of functionality should go, to make sure the components stay comprehensible. The first part of this series described an application as a group of interconnected entities. Now we will look at which entities to create and when.

There exist a number of MV* architectures, MVC, MVP, MVVM, … What these have in common is that the M (model) and V (view) have very clear and limited responsibilities and then everything else apparently goes into the * part. Display formatting of information, accessing the backend over the network, navigating through the application, you name it, the * handles it. And that’s just not enough for me.

Architecture with No Pronounceable Acronym

Over the years, I have come up with an architecture that works for me. The way that things are split into components makes sense, and I always know how to split up a feature and where each piece of needed functionality goes. I have not tried to force it to form any specific acronym, so it’s really just my architecture, with components created where they make sense.

I also like to separate the connections between components into two different kinds: A dependency means that a component has a reference to another component. And data flow means that a component passes data to another component. Sometimes data flow follows a dependency, when data is passed in to a component whose functionality is invoked. And other times data flow goes in the inverse direction, when a component observes another component’s state.

The diagram shows the useful architectural concepts that I have identified, which I call roles. The data flow between them is shown with arrows pointing in the direction of where the data flows. I have found that mostly, unidirectional data flow works well and makes things easier to understand, since it is always known from where changes can be triggered. Dependencies in the diagram go from top to bottom: something higher up can depend on anything lower down, but not vice versa.

Overview of Roles

It is of course necessary to have good definitions of the roles, to serve the goal of knowing where each piece of functionality belongs. View and Resource are the roles that interact with the outside world. A View is whatever draws the user interface on the screen and recognizes user interactions. A single screen often contains multiple Views, usually arranged hierarchically in containment relationships. A Resource is anything interacting with the external world outside of the user interface, from communicating with a backend to file I/O and even to timers.

Presentation and Control are both related to the user interface. In the diagram they are marked as view-local, whereas other roles are global. This means that usually each View has its own components to perform these roles, and they are not typically shared between different Views. The reason for splitting these roles is to make the data flow clearer (and unidirectional): Presentation is only concerned with how data is displayed on screen, so the data flow is unidirectional from Store through Presentation to View. And in the other direction, Control only receives user interaction triggers from the View and triggers actions in Store or Agent.

The division between Model and Store comes from how state is represented according to the previous post in this series. Models are values, and Stores are entities whose state consists of Model objects. All base state in the application is state of a Store. The Store can acquire this state either from data that flows into it from above, or by fetching it from a Resource, such as a backend or database. The data that flows out of a Store are the Model objects that comprise its state.

The Agent role is perhaps the most unusual and also the rarest. The idea behind Agent is that other roles are purely reactive; they do something only in response to data flowing into them, whereas an Agent is allowed to be active, to initiate an action (in a way, the View can also be seen as active, since user actions are definitely not initiated as a direct response to data flow, but user activity is not modelled here). For instance, a periodic synchronization of data with a backend would be an Agent activity, and separating the periodic triggering of the action by the Agent from the actual communication performed by the Store makes the whole system easier to comprehend.

Roles instead of Components

The key point in the diagram is that these are not architectural components, they are roles. The difference being, a component may assume multiple roles. When a Model object is shown in the user interface by simply displaying some of its properties as they are, I don’t bother to create a completely new Model Presentation component but simply let the View directly use the Model object that it acquires from a Store. But when the needed presentation becomes more complex, as it often does, the established separation of roles brings easily to mind the solution of creating a new component for the Presentation role.

I place great importance on being able to unit-test as much of an application as possible, and that in part has guided the architecture. The two roles marked as not unit-testable in the diagram are always separate components to facilitate this. For Resources, a test double is always used. Sometimes the double is a stub when the Resource interaction in the test is simple, sometimes a fake to allow more precise control over its behavior. Views are not considered in unit tests at all. All the logic determining what to show is in a Presentation, and all action code is in a Control, so there is no unit-testable logic in a View, only the actual displaying on screen and mapping user interactions to Control triggers.

Other Architectures

The architecture described here owes a lot to my experiences building mobile applications, and especially to using the popular architectures and trying to fill their “deficiencies”. For instance, the view-local part of the architecture comes quite directly from MVVM, where the View Model fulfils the roles of Presentation and Control. Experience with MVVM has taught me that these two roles are typically very distinct in a View Model, and for me separating them as concepts makes the whole architecture clearer.

Traditionally, the whole functionality contained in the Model, Store, and Resource roles would be contained in the Model part in an MV* architecture. The previous post on values and entities establishes why I like the Model-Store separation, and it is also partly inspired by the Model-View-Controller-Store architecture. The separation of Store and Resource is purely to help with unit testing. By separating the part that in a real application requires interaction with the external world into a thin stubbable layer allows unit tests to cover a lot more of the application than if all the Resource use was contained inside the Store.

Navigation is one of the most difficult architectural problems in mobile application development. It is always very simple at the beginning, and can be handled by the Views using the platform native navigation patterns. But too often the navigation patterns of an application become more complex, and they typically do this a bit at a time, never allowing the moment where it’s clear that the navigation needs to be rethought. The VIPER architecture is the only one I know that explicitly includes navigation, but even there it appears to be only a way to separate the too-simple platform-provided navigation into its own components. The kinds of complex navigation that I have encountered require a more global approach, usually requiring a single component that is responsible for the navigation needs of the whole application.

Scenarios

I began with the purpose of the architecture being to determine where each piece of functionality belongs. Thus, the validation of an architecture is performed by observing how well this purpose is fulfilled in development situations. The following two examples show how the functionality in an application screen would be split, and how the role-based thinking allows the implemented architecture to remain simple when applicable and to evolve into something more complex when needed.

The component architecture of a very basic screen could look like the User Info Screen example shown below. This is a component diagram, with the roles that each component fulfils shown on the top-right corner of the component. The User Store holds the User model object. The screen shows directly the information in this model object without any additional presentation logic, so the User object can act also in the Presentation role.

The screen allows editing the user information. In this case no special interaction logic is needed, so the User Store can act in the Control role for the View. Finally, the User Store is responsible for local and remote persistence of the user information, and it uses two Resources for this, one for the local database and one to communicate with the backend.

A more complex screen, with more components and more separated roles, is provided by a chat application. The Chat Store is in a way the center point of the application, as it stores all the chats that the user participates in. The Model objects are the messages in the chat, but in this case, a separate Presentation component for messages is needed; for instance, the Model object is likely to store the precise timestamp of the message but the screen should display a more human-friendly representation.

Depending on how message sending is implemented, there might be a Message Sender component in the Control role that both requests the Chat Store to send a new message from the user and immediately calls the Presentation to update the screen for a better user experience. An alternative would be for the representation of a chat provided by the Chat Store to also provide sufficient information for any pending messages. In general, a separate Control component is useful when a user action requires data to flow into several different components, but it could also be used when the Control functionality is complex enough to warrant testing but is not considered to be the responsibility of a Store.

The chat application also provides an example of the Agent role. The Chat Updater is an independent component that could be triggered by a timer, or by external notifications, either one of which would be a Resource. When triggered, the Chat Updater will tell the Chat Store to update a specific chat or all the chats using the Resource for communicating with the chat backend.

Summary

To summarize the above: I split an application architecture in as many concepts as I feel are necessary. I don’t feel the need to force them into a cutesy acronym. I use components only when they actually have some necessary function instead of being just a pass-through layer for data, and that’s why I think in terms of roles instead of components. I find thinking in terms of data flow very useful, so data flow between components is an important part of the architecture. And finally, I’m not afraid to go against my principles when needed. Not everything fits into neat little boxes, and sometimes it’s better to just acknowledge that than to try to squeeze a square peg into a round hole. In the end, software architecture should provide guidance, not act as a straitjacket.

Programming with State: Values and Entities

Jaakko Kangasharju — Wed, 14 Aug 2019 14:38:25 +0000

Developers know well the perils of mutable state in a program. A value changes from underneath you, messing up a computation in progress. Two different places on the screen, ostensibly based on the same information, are not consistent with each other. The application crashes because a value was not changed at an appropriate time.

The functional style of programming, which is based on immutable values and functions that build new values instead of mutating existing ones, has become very popular due to the complexity of managing state when everything is mutable. But let’s say that we find programming with mutable state useful. For instance, my experience is primarily in mobile app development with the native tools, where the whole platform is built around mutable state, and it is a rare team or company that is willing to go for a completely different app architecture or programming style than is practiced by the majority of developers. The main question then becomes how do we manage that state to keep our programs comprehensible.

Values and Entities

A view that I’ve found useful is to divide the objects manipulated by the program into two kinds, values and entities. A value is conceptually similar to a primitive type, an immutable object, with a structured value simply a sum of its parts with no hidden content. In contrast, each entity is a distinct object, even from entities with the exact same state. And as an entity has a distinct identity, its state can be mutable.

Value	Entity
State-based equality	Distinct identity
Immutable	Internal mutable state
	External observable state
“Functional”	“Object-oriented”

This view conflates equality with mutability, but in my opinion the other possibilities are not needed. An immutable object with a distinct identity can simply be a value that includes a unique identifier. And a mutable object with state-based equality is in my experience just asking for trouble.

Languages like C# and Swift have some of this separation at the language level, with the separation of types into structs (values) and classes (entities). But they don’t go all the way, as both also include an overridable equality for their entity types, allowing a non-identity-based equality to be defined. I feel like this is a holdover from languages where “Everything is an entity” and has no place in a proper value-entity-separating language. The “identity has a state” approach of Clojure is much better but we can’t always program in Clojure.

I’ve used the terms “Functional” and “Object-oriented” in the table above, as this is how a programming style primarily based on one kind feels to me. A functional style is based on values, building new values out of pure computations on existing values, whereas an object-oriented style is based on distinct entities having internal state modified by their external interfaces. The rest of this post will be focused on the entity-based object-oriented style relying on mutable state, so I won’t say more about the pure functional style.

State

Now that the big beast of mutable state has been pushed into these things I call entities, let’s look at how to manage it. I’ve split state in an entity into two forms, internal and external. It doesn’t matter much how internal state is managed, as that is local to the entity and can be contained in a small enough area to usually be understandable.

For external state, it should make sense as something that is exposed by the entity. Most often, it is better not to simply expose a piece of internal state directly as external state, but provide something more meaningful to the outside world. Or even expose only invocations that can be performed on the entity, whose implementation depends on the internal state but no specific state is ever exposed.

Thinking in terms of invariants helps keep the state of an entity manageable. An invariant is simply a property that holds among different parts of an entity’s state. For instance, in an entity representing a shopping cart, the total price should be the sum of the prices of the individual items minus any applied discounts. While an entity is modifying itself, an invariant can temporarily be violated, but an external observer should always see the invariant to hold.

I used the term “observable” in the table above for external state. This, to me, is necessary to manage the application state properly. It must be possible for other entities to observe changes in an entity’s state and to react to those changes. In a stateful application, pieces of state are never fully independent of each other, so keeping the complete application state consistent requires modifying many pieces of it simultaneously.

Dependencies

Observable state and reacting to its changes introduces the notion of an entity, and its state, depending on the state of another entity. Internally, entities can contain references to other entities that they depend on. And their state can depend on the state of other entities. So there are two kinds of state: Base state and derived state, where derived state is a composition of other state.

I find that a good way to manage this is to represent the observable state of an entity as values. Following this, derived state can be defined as a pure function of its dependent state. This is essentially how spreadsheets work, and that is a very useful conceptual model for understanding the complete application state as consisting of independent pieces of base state and dependent pieces of derived state. Entities may contain other entities as internal state but these contained entities are never exposed as external state.

To do this properly, the programming language or environment needs to support some form of data binding. Derived state is defined by binding to all its dependencies, with a pure function to compose it. This way the system guarantees that all the state throughout the application is always consistent. This can in theory be also done with the Observer pattern, but as the Observer pattern splits the definition of the state from the update, in practice it can be too easy to forget some necessary update to derived state when the relationships get complex.

The Redux architecture shows similar thinking. State changes are performed with pure functions that take the existing state and a performed state-changing action. Redux essentially has only one entity, the store, which simplifies derived-state management. When combined with something like React, where the UI is a pure function of application state, the complexity of state management is significantly reduced.

In platforms that encourage the multiple-entity style, derived-state management is not really supported all that well in languages themselves or their libraries. Data binding is usually treated through modification events, which do not function well in, say, the diamond dependency scenario: The state of entity Top depends on the states of entities Middle1 and Middle2, both of which depend on the state of entity Bottom. The correct order to modify the states is Bottom -> (Middle1, Middle2) -> Top, which is followed by any spreadsheet, but a system based on modification events will do Bottom -> Middle1 -> Top, where the new state of Top is computed based on the new state of Middle1 but the old state of Middle2, leading to an inconsistency.

For large pieces of state, it is not really the best idea to rebuild all depending state from scratch when something little changes. For instance, if the state represents the whole product catalog of a shop service, adding one more product is a very small change that should not trigger massive recomputations. In such situations, I usually include an additional stream of change events. A change event is essentially just the difference between the previous state and the current state, and it has enough information so that any derived state can be modified to match the new dependent state. So when first creating an entity with derived state, a full computation is performed, but afterwards, the derived state is modified in place to reflect changes communicated through the change events.

I Am Unhireable

Jaakko Kangasharju — Tue, 09 Jul 2019 12:28:12 +0000

Following discussions around programmer recruitment, I often come across people who talk about simple criteria they have for who could even get an interview. And when thinking about such criteria, I typically discover that I don't fit them. I guess all this makes me unhireable?

(Phrases in quotes are paraphrases of the common attitudes I see, not quoting any specific people.)

I am in my mid-40's

"Programming is for young people." I didn't even become a professional developer until my mid-30's, though my work before that did involve coding. Granted, I don't think my mind is as quick as it was, but that also helps me avoid making my code too complex. And I do have quite a bit of experience, and even experience in now-obsolete platforms can turn out to be useful.

I've worked in the same place for 10 years

"I am suspicious of anyone who's been in the same place too long." "2-3 years is the optimal length of one job." I don't really see any point in making a change just for the sake of making a change. I like my job, I like the company, and I've had the opportunity to develop myself in interesting and challenging directions.

I learned to program in the university

"All good developers started young, playing with their childhood computer." We did have a computer when I was a child, but I mostly used it to play games. I knew some BASIC but whatever I did with it wasn't much help later on. I didn't even get the point of subroutines. Sure, it probably helped me become comfortable with computers in general, but it's not like I did anything of the sort that people tend to expect.

My Github is small and irrelevant

"I always look through a candidate's Github and expect to find a professional portfolio." Here are my Github contributions

In fact, the period shown here has been exceptionally busy for my Github.
Furthermore, practically everything in my Github is small side projects, not showing anything of my actual professional work. Anyone going through my Github is going to have a very distorted view of my skills.

I don't have a "passion" for coding

"Good developers spend their free time coding, they cannot stop." I never feel like coding just for coding's sake. I occasionally have side projects, but those are for real problems that I want to solve where the solution involves coding something. I do like coding, which is good since it's my job, but when I finish at work, I tend to do something else on my free time.

I have a PhD

"I want to hire practical people, not ivory tower academics." This is perhaps not a bad thing everywhere. But in Finland, my native country, it definitely is. Fellow students of mine who worked in the industry hid their PhDs at work to avoid being pigeonholed. And recently I've been getting a feeling that there is beginning to be more sentiment against more academic backgrounds in general in software development.

Nevertheless...

Despite all these ways in which I am "unhireable", I'm still employed, still well-respected at work, and consistently get great feedback on both my programming and social skills. So maybe making snap judgments on one characteristic is not the best way?

It does make me sad when my colleagues talk about using some of these criteria or similar ones. I've gotten into the habit of saying "Doing that would mean you wouldn't hire me. Is that what you want?" but, as one could expect, it doesn't really cause them to re-evaluate their thinking all that much.

Finally, of course all that is irrelevant since I possess the most sought-after skill of a developer: I am excellent at coding algorithms on a whiteboard.

Tennis and Amplifying Small Differences

Jaakko Kangasharju — Tue, 04 Jun 2019 13:19:32 +0000

A while ago there was a nice post by @teckert on probabilities in a basketball game

Basketball for Nerds

Thomas Eckert ・ Apr 10 '19

#math #probability #interviewquestions #python

Generalization

Thinking about the two situations in the post I came up with the following generalization: You have an odd number of chances and you have to make more than half the shots. This fits both the "1 out of 1" and "2 out of 3" cases that were in the post. And it lets us visualize the effect that @walker pointed out in a comment, that the more chances you have, the more the final result of the game is determined by your individual shot probability.

In the original post, @teckert computed the probability in the "2 out of 3" game by hand. But if we want to generalize to really long games, the formulas would be really unwieldy to derive and handle. We could also write a loop that would generate @teckert's table, letting the computer handle the heavy lifting. Is there a better way?

Turns out, there is. We have a situation where we have n chances and probability p of making each of them, independently of how the others have gone. This means that the number of successes follows the binomial distribution. And SciPy has a ready-made function for computing this.

We want to determine the total probability of all the cases where the made shots are more than half the number of chances. In probability, such a calculation can be done using the cumulative distribution function (often abbreviated as CDF). The CDF gives the total probability of getting a value less than its argument, so to get the total probability of getting a value more than half, we can subtract the CDF value from 1 (note that we're assuming an odd number of chances so it is not possible to make exactly half the shots).

from scipy.stats import binom

def winning_probability(number_of_chances, shot_probability):
    return 1 - binom.cdf((number_of_chances - 1) / 2, number_of_chances, shot_probability)

Equipped with this function, we can plot the game winning probability curves for various numbers of chances.

import matplotlib.pyplot as plt
import numpy as np

plt.figure(figsize=(10, 6))
p_range = np.linspace(0, 1, 100)
for n in [1, 3, 5, 9, 19, 39, 79]:
    probs = [winning_probability(n, p) for p in p_range]
    plt.plot(p_range, probs, label=str(n))
plt.ylabel("Probability of Winning Game")
plt.xlabel("Probability of Making a Single Shot")
plt.legend()
plt.show()
display()

The resulting graph shows how, the more chances we have, the more the probability approaches just 0 or 1, depending on whether we're worse or better than average.

The Case of Tennis

So what does this have to do with tennis? Well, tennis shows a similar amplification of small differences. But in tennis, the effect happens on multiple levels. First, you need to win four points to win a game, you need to win six games to win a set, and finally you need to win three sets to win the match. If you lose a game or set, it doesn't matter further on how much you lost by.

Tennis also presents a complication. To win a game it's not enough to win four points, you also need to be two points ahead of the opponent. So we can't use the binomial distribution anymore. But we can set up an equation for the probability and solve it. The key is noticing that, when the score is tied with two consecutive points needed to win, there are three possible situations

You win two points in a row
Your opponent wins two points in a row
You both win one point of the next two

In case 1, you win the game. In case 2, your opponent wins. And in case 3, the situation ends up back in the beginning. So we can set up a formula for the total probability, which will be a sum of two terms: the probability of case 1 and the probability of you winning in case 3. These work out to

case_1_probability = point_probability * point_probability
case_3_probability = 2 * point_probability * (1 - point_probability) * total_probability
total_probability = case_1_probability + case_3_probability

This doesn't exactly work out as computer code because of the recursive definition (total_probability depends on case_3_probability which in turn depends on total_probability), but we can expand the definition of total_probability to make a perfectly solvable mathematical formula

t_p = p_p * p_p + 2 * p_p * (1 - p_p) * t_p

Standard algebraic manipulations produce the solution

t_p = p_p * p_p / (1 - 2 * p_p * (1 - p_p)) = p_p * p_p / (1 - 2 * p_p + 2 * p_p * p_p)

which we can implement in code

def two_wins_in_a_row(point_prob):
    return (point_prob * point_prob) / (1 - 2 * point_prob + 2 * point_prob * point_prob)

So this is the probability of winning a game when you and your opponent both have, say, 3 points, and you need to win two points in a row.

Let's start putting things together for computing the probability of winning a game when we know the probability of winning a point. Conceptually, we will fill a table indexed by your and your opponent's score with your winning probability from that score. Then, the probability of you winning the game is simply the value in the table cell (0, 0).

O\Y	0	1	2	3	4	5	...
0					1	1	...
1					1	1	...
2					1	1	...
3				T		1	...
4	0	0	0		T		...
5	0	0	0	0		T	...

We can start by filling in the value 1 in each cell where you have 4 or more points and your opponent has at least 2 points less than you, since you've already won those. Similarly, we fill in 0 in the symmetrical cases where your opponent has already won. And finally, for 3-3 and even scores after that, we fill in T to signify the value is computed directly by calling two_wins_in_a_row.

We want to end up with a function signature like

def game_win_prob_with_scores(point_prob, score_you, score_opponent):

which will return the value in the table cell indexed by score_you and score_opponent. Then the probability of you winning a game will be game_win_prob_with_scores(point_prob, 0, 0). From the table that we filled in, we can already implement some simple cases

    if score_you >= 4 and score_you >= score_opponent + 2:
        return 1
    elif score_opponent >= 4 and score_opponent >= score_you + 2:
        return 0
    elif score_you >= 2 and score_you == score_opponent:
        return two_wins_in_a_row(point_prob)

(We noticed that in fact T is the probability already when the game is 2-2)

How to handle the remaining cases? We notice that if the game is now at score x-y, the probability is point_prob that it will next be x+1-y, and 1 - point_prob that it will be x-y+1. So if we know the values in the table right and below some cell, we can compute the value in that cell from those other two values. And in the table we see the "border" going from 4-0 to 4-2 to 3-3 to 2-4 to 0-4 that will let us eventually compute any value to the top and left of it.

So now we can write the code for the else

    else:
        return point_prob * game_win_prob_with_scores(point_prob, score_you + 1, score_opponent) + (1 - point_prob) * game_win_prob_with_scores(point_prob, score_you, score_opponent + 1)

Because the border in the table is implemented by the first three conditions in the if-elif chain, and both recursive calls go towards the border, the recursion will eventually terminate (always make sure recursion terminates, especially in cases like this where it's not so obvious at first glance).

There is still one problem with the code. Say we start from score_you == 0 and score_opponent == 0. The left recursive call will have parameters score_you == 1 and score_opponent == 0 and the right recursive call score_you == 0 and score_opponent == 1. The problem is that from both these calls, the function will be called with parameters score_you == 1 and score_opponent == 1, so there will be duplicate work.

This could be solved by rewriting the function to be non-recursive and to process the table cells in an order so that each cell is computed only when its dependencies to the right and down are known. A simpler way is to apply memoization to the recursive version. This is especially easy in Python, since there is a decorator in functools that converts any function into a memoized version. So we add

from functools import lru_cache

at the top, and

    @lru_cache(maxsize=None)

before the function game_win_prob_with_scores and we're done.

From the game-winning probability we can use the same technique to convert it into a set-winning probability. The winner of a set is the first player to reach 6 won games with the other having at most 4, or reach 7-5, or win the tie break at 6-6. The tie break is the same as a game, except it's played until 7 points.

The match is easiest to handle by special-casing the three different possibilities, 3-0, 3-1, and 3-2.

def match_win_prob(point_prob):
    set_prob = set_win_prob(point_prob)
    three_nil_prob = set_prob ** 3
    three_one_prob = 3 * set_prob ** 3 * (1 - set_prob)
    three_two_prob = 6 * set_prob ** 3 * (1 - set_prob) ** 2
    return three_nil_prob + three_one_prob + three_two_prob

The multipliers 3 and 6 count in how many ways each type of win can happen. For instance, in a 3-1 win the 1 set won by the opponent can be the first, second, or third.

And this brings us to the final graph, of showing how the rules of tennis amplify even small differences in point-winning probability to make it much more certain for even a slightly stronger player to win a match.

This was of course somewhat simplified, since in real tennis the probability of winning a point is not constant. Especially having the serve is a major advantage. It would be possible to perform a similar analysis, considering two different probabilities, but that would be unlikely to demonstrate any additional interesting point.

Appendix

Here is the full code for the tennis example, refactored to take advantage of game and tie break being exactly the same, except for a different target number of wins.

import matplotlib.pyplot as plt
import numpy as np
from functools import lru_cache

def two_wins_in_a_row(point_prob):
    return (point_prob * point_prob) / (1 - 2 * point_prob + 2 * point_prob * point_prob)

@lru_cache(maxsize=None)
def simple_win_prob_with_scores(point_prob, target, score_you, score_opponent):
    if score_you >= target and score_you >= score_opponent + 2:
        return 1
    elif score_opponent >= target and score_opponent >= score_you + 2:
        return 0
    elif score_you >= target - 2 and score_you == score_opponent:
        return two_wins_in_a_row(point_prob)
    else:
        return (
            point_prob * simple_win_prob_with_scores(point_prob, target, score_you + 1, score_opponent)
            + (1 - point_prob) * simple_win_prob_with_scores(point_prob, target, score_you, score_opponent + 1)
        )

def tie_break_win_prob(point_prob):
    return simple_win_prob_with_scores(point_prob, 7, 0, 0)

def game_win_prob(point_prob):
    return simple_win_prob_with_scores(point_prob, 4, 0, 0)

@lru_cache(maxsize=None)
def set_win_prob_with_scores(point_prob, score_you, score_opponent):
    if (score_you == 6 or score_you == 7) and score_you >= score_opponent + 2:
        return 1
    elif (score_opponent == 6 or score_opponent == 7) and score_opponent >= score_you + 2:
        return 0
    elif score_you == 6 and score_opponent == 6:
        return tie_break_win_prob(point_prob)
    else:
        game_prob = game_win_prob(point_prob)
        return (
            game_prob * set_win_prob_with_scores(point_prob, score_you + 1, score_opponent)
            + (1 - game_prob) * set_win_prob_with_scores(point_prob, score_you, score_opponent + 1)
        )

def set_win_prob(point_prob):
    return set_win_prob_with_scores(point_prob, 0, 0)

def match_win_prob(point_prob):
    set_prob = set_win_prob(point_prob)
    three_nil_prob = set_prob ** 3
    three_one_prob = 3 * set_prob ** 3 * (1 - set_prob)
    three_two_prob = 6 * set_prob ** 3 * (1 - set_prob) ** 2
    return three_nil_prob + three_one_prob + three_two_prob

p_range = np.linspace(0, 1, 100)
probs = [match_win_prob(p) for p in p_range]
plt.figure(figsize=(10, 6))
plt.plot(p_range, probs)
plt.ylabel("Probability of Winning Match")
plt.xlabel("Probability of Winning a Point")
plt.show()
display()

Detecting Cute Animals with Machine Learning

Jaakko Kangasharju — Wed, 17 Apr 2019 00:17:39 +0000

Training and building a custom image classifier mobile app

Photo by Álvaro Niño on Unsplash

When a data scientist colleague of mine recently found out I have a background in mobile app development, he asked me to show him how to use a machine learning model in a mobile app. I figured an image classification app, like the classic Not Hotdog, would be a nice example, since it also requires hooking into the phone’s camera and is not just a model running on a phone instead of desktop.

It took me a while to piece together everything that I needed to make a finished app. That’s why I thought writing a post would be useful, so that the full journey is documented in one place. All the code that I wrote, fetching the training images, training the model, and the app itself, is available on Github. The code is comprehensively commented, since I wanted it to be useful for both data scientists and mobile app developers.

What to Build?

At first I thought I’d just build a hot dog detector, and found a post that went through the model building part. But I don’t have many hot dogs around, so demoing such an app would be difficult. Since I like cute animals, I decided instead to make a cuteness detector, which, as a more abstract concept, can be demoed in many everyday environments.

For the source of training data, I picked ImageNet. I decided to take puppy and kitten pictures as my “cute” training data, and somewhat arbitrarily, creepy-crawlies and ungulates as my “not cute” training data. After filtering out non-available images, I was left with 1718 cute images and 1962 non-cute images. Verifying visually that I had good data definitely validated the choice of creepy-crawlies as “not cute”. Brrrrr…

Transfer Learning with Inception

For custom image classification, transfer learning is the way to go. Training an image classifier from scratch would take ages and any network architecture I could come up with couldn’t be better than current state of the art. I picked InceptionV3, since I found a good article with example code of how to make a custom image classifier with it.

The key points in building the model are to take the InceptionV3 model as base, add a new classification layer, and limit the training to the new layer. Since the training data comes from ImageNet, the InceptionV3 constructor is told to use those weights. The other parameter says to exclude the top layer, which does the actual classification, since I will be using my own layer for that.

Nowadays, image classification has moved away from using pooling after each convolutional layer and to using global average pooling at the end. So the base InceptionV3 model is extended with a GAP layer, and finally a softmax classification layer with two nodes, since there are two classes to predict, cute and not cute. In the final step, all the layers from the base InceptionV3 model are marked as not trainable so that the training applies only to the added custom layers.

Photo by Liudmyla Denysiuk on Unsplash

Data for Training

I had downloaded my training images, and now there is a little bit of work to prepare them for training. A good way to set up data for Keras is to make directories for each of training, validation, and testing data sets. Then, in each of these directories, create a subdirectory for each label and put the images for each label in the corresponding subdirectory. So I have directories “cute” and “notcute”. This lets Keras automatically pick up the labels during training.

Augmenting the data is always a good idea when doing image classification. Keras comes with the ready-made ImageDataGenerator that can be used to automatically augment image data with different transformations.

The ImageDataGenerator is initialized with parameters on which kinds of transformations it should perform. I didn’t pick any that would introduce distortions, since a distortion might well remove cuteness. InceptionV3 expects the pixels to be in the range [-1, 1], and it provides the function preprocess_input to perform the min-max scaling to this range. This function is passed to the generator so that the generated images will be in the appropriate format.

The generator is then set to produce the images from the image directory. The parameter dataset can be either “train” or “test”, depending on whether to generate training or validation data. The image size 299x299 is what is expected by InceptionV3.

Building the Model

Training the model with the generators is simple enough. Five epochs appears suitable for this task. In fact, I didn’t see improvement after the third epoch. With these parameters, the training on my Macbook took about 2.5 hours, which I felt was quite reasonable.

The last line above, model.save('iscute.h5'), saves the model in the HDF5 format used by Keras. This is not the format used for mobile apps. For Android, I chose to use tensorflow-lite. (I wanted to make an iOS app using CoreML, but I was using Python 3.7, and the CoreML tools do not yet support that version.) Tensorflow comes with the appropriate converter, so there is not very much code here. The HDF5 file I got is 84 MB in size, while the tensorflow-lite file is 83 MB, so there was not much size reduction.

The only headache here was discovering the need for the input_shapes parameter. On my initial run without that parameter, I got the error message “ValueError: None is only supported in the 1st dimension. Tensor 'input_1' has invalid shape '[None, None, None, 3]'”. The required dimensions are the image dimensions 299x299, and the error message says what tensor name to use as the key in the dictionary parameter.

Photo by Ricky Kharawala on Unsplash

Reading the Camera on Android

Now I had my model ready to use in the app, so it was time to build the app. It needs to do the following:

Show the camera preview on screen
Let the user press a button to capture the current image
Feed the captured image to the classifier
Display the result after classification is complete

The parts I didn’t yet know how to do were 1 and 3. Furthermore, the Android camera API had changed, and many tutorials I could find were still written for the old, deprecated version. Luckily I did find one post that detailed everything that needs to be done. There is quite a bit of code needed to access the camera image, so I won’t show most of it here. Check the repository for the full code.

To prepare for image capture requires getting a CameraManager, asking the user for permission to use the camera (unless already granted), and getting a reference to a TextureView that will contain the preview. A real app would need to prepare for the camera permission not being granted as well.

The camera cannot display anything until the TextureView is available, so the app may have to wait for that by setting a listener. Afterwards the app needs to find an appropriate camera (the back-facing one here) and open it.

The openCamera() method passes in a state callback. When the camera is opened, this callback will be informed, and it can trigger creating a preview session with the camera so that the camera image is available when the user presses the evaluation button.

Photo by Jairo Alzate on Unsplash

Making Classifications on Android

To figure out how to make the classifications, I briefly looked at the Tensorflow-lite guide. While it helped to get started, it was much less help with the actual details of the code. For the details, I went through the sample app that does image classification, and extracted out the specific bits I needed out of the very general code.

Initializing the classifier means loading the model into memory from the app’s assets, and creating the Tensorflow Interpreter object out of that. The input for an image classifier is usefully given as a ByteBuffer and it’s so large that it makes sense to pre-allocate it during initialization.

When running the classifier on an image, it’s important to perform it in a background thread. It takes a few seconds to run, and holding up the UI thread for that long is simply not acceptable. The first thing to do is to create the input in the correct format in the buffer: Scale the image to 299x299 and run the min-max scaling to get the pixel values to the range [-1, 1], as expected by InceptionV3.

And finally, the classifier can be run. The type of the run() method is not very helpful, as it just takes an Object as both input and output. As mentioned above, for image classification, ByteBuffer is a good input format. The output in this classification case is a one-element array, the element of which is an array of floats whose size is the number of classes. After the run, this array of floats will be filled with the probability of each class. So in my case, it is a two-element array, with the elements being the probabilities of “cute” and “not cute”.

I did not find a way to determine which class corresponds to which index in the array at this stage. So the class-to-index mapping would need to be extracted from the Keras model, and possibly included as data in the app, in case the mapping is not stable between different training runs.

Summary

In the end, it was not too difficult to build an image classifier app. It’s cross-cutting across a few areas, so the information is not all in one place, but when you look, it’s all available somewhere. The app I built is of course only a demo, but knowing the principles and methods now, I am confident that I could add this kind of functionality in a real app too.

Screenshot of the app evaluating a dog

Yep, I think my model works well.

My Developer Interviewing Principles

Jaakko Kangasharju — Wed, 10 Apr 2019 12:21:42 +0000

Photo by Kaleidico on Unsplash

As a senior developer, a part of my job is giving technical interviews to candidates for developer positions at Futurice. Developer interviewing is quite a well-discussed topic around the Internet, and over time, following these discussions and thinking what I want to see, I’ve developed something of a style. For this post, I’ve attempted to distill this style into a set of principles that I try to follow whenever I’m interviewing.

An Interview Is Not an Exam

In an interview, I am evaluating your developer knowledge. This is similar to an exam, which is also about evaluating your knowledge. But an exam is limited to a specific area of knowledge, which makes it possible to “cram” for it. An interview should cover a much wider area than an exam, and also be more interactive than a typical exam.

In fact, I think if it’s possible or necessary to cram for an interview, it’s a bad interview. My task in the interview is to determine whether the candidate has the skills that my company needs of a developer. If I give an interview that can be crammed for and “passed” without possessing the needed skills in reality, I’m making a potentially costly mistake.

This principle I followed already when I was first looking for a developer position. I never prepared for the interviews. I just went in and was myself, answering questions based on what I could think up on the spot, not giving prepared answers. And I feel like I would not want to work in a company where that method does not work.

An Interview Is Not an Interrogation

Some interviewers seem to treat the interview like an interrogation. They ask questions in a rapid-fire style, expect only short answers, and work off a specific list that they have. This is not how I like to do things. I try to get you to talk most of the time, as I feel that’s how I can get the best evaluation out of you. And I try to pick up specific points from your answers and ask follow-up questions about those, making the interview more of a discussion than a Q&A session.

That doesn’t mean I don’t have a list of prepared questions. Working off a specific list, with good ideas of what kinds of answers are expected, helps ensure that the evaluation is performed similarly for different candidates. The difference is, my questions don’t usually have simple answers, so they serve more as discussion starters, with potential for following up to whatever level you’re proficient at.

There Should Be No One Right Answer

Software development is not an exact science. And even if it was, the amount of variables is so large that even very similar problems can call for very different solutions. It is very rare that a practical question could have just one answer that is correct every time. So I prefer open-ended questions with many possible solutions to questions where I would be hunting for a very specific response.

Yes, this means I won’t be asking specifics about language syntax or libraries. I do expect a developer to have a working knowledge of these, but that knowledge is better demonstrated by writing code, not answering questions about them. When I care, say, about some library functionality, I prefer to introduce the general problem, discuss that, and see if you’re aware of the common libraries or methods to solve that problem. I won’t expect you to know the precise functions to call off the top of your head, let alone their syntax, but awareness of the functionality could be expected.

My Opinions Are Not the Only Correct Ones

Even keeping in mind the previous point, we developers do tend to pick favorites. For common problems, we have specific solutions that we tend to reach for, assuming they’re not infeasible. I acknowledge that both I and you will have these favorites, and we may have picked different ones. As long as you show familiarity with your solution, including its potential drawbacks, I won’t hold it against you if we didn’t pick the same one.

Typically, you won’t even find out my opinion on whatever I asked about. Knowing what I like can easily bias you to giving that answer even if you prefer something else. And that will tell me less about how you think. So I try to keep my questions neutral, I try to be aware of the drawbacks of any of the potential solutions and ask about how you would address them.

Sometimes I Ask Bad Questions

I don’t actually have a principle to ask bad questions occasionally… But we’re all human, and no matter what lofty principles I try to follow, I will make missteps. The important thing for me is to recognize I asked a bad question, put the blame on myself and not penalize you for not comprehending what I was going for, and learn from it for the future.

The most important thing, when I notice I asked a bad question, is to put you back at ease. Not being able to answer an interview question can easily feel like you’ve failed. So I won’t just move on. I will try to rephrase what I was asking, clarify if needed, and if it seems appropriate, admit that it wasn’t a good question. We’re not robots, and it’s good for me as an interviewer to also admit I can make mistakes.

I Don’t Care How Well You Function under Pressure

A job interview is a stressful situation. But that’s not what the work itself should be like. If you’re constantly under stress at work, there is something wrong in the environment that should be fixed. So putting you under pressure in the interview is not going to tell me anything useful about you. That means, for instance, no live coding. It wouldn’t be a typical environment for you, and I think the time can be spent better.

Keeping in mind all the points made above should also help in relieving the pressure. When I treat the interview more as a discussion, ask questions prompted by what you say, and don’t make you recall minutia, that usually contributes to making you more at ease. But I still need to observe, and try to adapt if you begin to look stressed, because not everyone is the same, and not everyone has the same pressure thresholds.

But I Do Want to See Your Code

To be hired as a developer, you should know how to code. As I said above, I don’t think making you code in the interview is going to show that usefully. But I still need to evaluate your coding ability. So that means, you need to provide some code of yours for me to see. And this should be done before the interview so that I can consider it properly.

I don’t care so much where the code is from. It can be something you’ve already done, or it can be a coding exercise specifically for the interview made on your own time. Whatever suits you best. I won’t require you to have an extensive existing open-source portfolio ready to show, and I consider the calls to require such to be a bad idea in general.

Much of the interview will be spent on looking at your code together. I will look for typical good coding practices. I will ask you to explain why you designed the code the way you did. I will ask about functionality that you might have to implement next. And I will ask about specific pieces of code where I see multiple reasonable ways to accomplish the goal.

Because of what I want to ask about, my preference is for a short, 2–3-hour, specific coding exercise. Such an exercise would be defined to require functionality that we typically encounter in real projects, and I can ask about the same questions of everyone. Also, since you wrote the code shortly before the interview, it’s likely to be clearer in your mind. But as long as the code you provide has the interesting functionality, and is written to show your skills instead of being a quick hack, it should be just fine.

Fixing Regressions with Git

Jaakko Kangasharju — Tue, 12 Mar 2019 15:06:49 +0000

I'm sure many of us have been there. A bug has appeared and you're sure it was working a couple of weeks ago. But it's a big project, with hundreds of commits and thousands of changes since then. Luckily Git has some tools to help track down exactly when and why that breaking change was introduced.

`git blame`

The simpler tool is git blame. This can be used when you know which code is causing the problem, and you only need to find the commit to figure out why the change was made. For example, say the code in boggles.lang looks like this

refresh_button.enable()
frequency = compute_frequency(next_time)
baffler.aeronate(frequency)

and the problem is that refresh_button is disabled.

Now, perhaps you're familiar with the code. Maybe you even wrote most of this functionality. You go "Of course! Aeronating the baffler starts the norigator and the refresh button is disabled whenever the norigator is running." So you know that the problem is with the line

baffler.aeronate(frequency)

First Attempt

Of course you can't just go deleting this line. Someone added it for some specific purpose. This is where git blame comes in. You start with

git blame boggles.lang

and browse to the line

baffler.aeronate(frequency)

It will show up as something like

30376ff1 (Tomasz 2018-10-23 14:56:11 +0100  65) baffler.aeronate(frequency)

This line has the following pieces of information, in order:

The commit identifier
The author of the commit
The date and time of the commit
The line number
The contents of the line itself

What this tells you is the last commit that touched this line. That doesn't mean that it was the commit that introduced the problem. To check the commit, you would run

git show 30376ff1

which might tell you something like

commit 30376ff158bd8713fbf3308efcc6a46d0b9cdf57
Author: Tomasz <tomasz@company.example>
Date:   Tue Oct 23 14:56:11 2018 +0100

    Clean up trailing whitespace

This is not very helpful...

Look Only at Actual Changes

Whitespace changes are practically never what I'm looking for when trying to track down a regression, so that's why I use the -w option of git blame to ignore whitespace changes:

git blame -w boggles.lang

Now the output might look like this

3f056217 (Lucia 2018-10-19 10:22:41 +0100  65) baffler.aeronate(frequency)

Again going in with

git show 3f056217

and scrolling to the appropriate line in the diff might now reveal something like this

 refresh_button.enable()
-fq = compute_frequency(next_time)
-baffler.aeronate(fq)
+frequency = compute_frequency(next_time)
+baffler.aeronate(frequency)

This is a diff that shows only that the local variable fq was renamed to frequency but the issue-causing aeronation is still present. We must dig deeper.

Go Further Back in History

git blame has a nice feature where it can begin looking from any specific commit. We now know that the issue we're tracking was introduced before commit 3f056217, so we can run

git blame -w 3f056217^ boggles.lang

(The ^ is important; it tells to start the search from the parent of 3f056217. You can also use ~ instead of ^ to get the parent; the differences of those two do not matter in this case.)

This sequence is already familiar. Scroll to the appropriate line in the output:

7e3d2394 (Peter 2018-10-17 14:02:01 +0100  65) baffler.aeronate(fq)

Check the commit:

git show 7e3d2394

Scroll down to the code in the diff:

 refresh_button.enable()
+fq = compute_frequency(next_time)
+baffler.aeronate(fq)

And success! Now we have found the commit that introduced the problematic aeronation.

So what next? Now I would read the commit message of 7e3d2394, and potentially the whole commit diff, to determine what functionality the commit introduced, and try to figure out how to reconcile those needs with the needs of the refresh button to stay enabled. If I could not figure that out, I would likely have a chat with Peter who introduced the aeronation to see if together we could come up with something. A typical case might be that the aeronation is doing too much and does not in fact need to start the norigator in all cases. This would be handled by splitting the method into two and calling only the aeronation part here.

`git bisect`

What if you can't tell where the problematic code is? You can tell that the code has changed since it was first written, but you have no idea how aeronating the baffler could have anything to do with the refresh button, so that's not where your mind goes first. And there have been many changes here recently, so it's not that there's only one change to consider.

Setting up Bisection

Git itself can help you here. The git bisect command is made for exactly this situation. If you know a commit that is "good" and another later commit that is "bad", you just tell git bisect that, and it will let you look through commits between them until you zero in on the one that caused the change from good to bad.

The way I begin this process is by first running

git log --first-parent

and look for some commit where the feature was still working. Of course I can't tell that by just reading the log, so I mostly rely on the commit timestamps and my memory of when things were definitely fine.

(The --first-parent option makes git log show commits only on the current branch, not commits from other branches that have been merged. It's useful for this kind of case.)

Let's say I picked commit 7102f250 as a likely one where the feature was still working. Now I will do

git checkout 7102f250

and test this version. If I discover that the feature is not working, I will make a note that it was broken already on this version and go back in the logs until I find a working version.

After I have a working version, it's time to start bisecting:

git bisect start
git bisect bad master
git bisect good 7102f250

(If I discovered some earlier commit where the feature was already broken, I would place that commit's identifier in place of master on the git bisect bad master line.)

Evaluating Commits

After executing all these statements, there will be output something like

Bisecting: 105 revisions left to test after this (roughly 7 steps)
[3f056217e73576fc6ba8f0cb8c277627fd154e43] <Imagine useful commit message here>

This means that git has determined there are 105 possible versions that could have introduced the bug. It estimates needing 7 tries to determine the actual version (the search is binary-search-like so it needs roughly the binary logarithm of the number of possibilities). And it has checked out a suitable version to try.

Now that git has checked out a version to test, you need to do exactly that. Test whether this now-checked-out version has the bug or not. If it does, tell git

git bisect bad

If it works correctly, tell git

git bisect good

And if for some reason you can't determine whether the bug is present or not, you can do

git bisect skip

For instance, if the version has another bug that prevents you from getting to where your bug of interest is, you would need to use skip.

Continuing with this process, testing each version that git checks out and marking it as good or bad, should eventually end up with something like

7e3d2394e73576fc6ba8f0cb8c277627fd154e43 is the first bad commit
commit 7e3d2394e73576fc6ba8f0cb8c277627fd154e43
Author: Peter <peter@company.example>
Date:   Wed Oct 17 14:02:01 2018 +0100

    Make sure baffler is aeronated...

Now we found the same commit that we found earlier with git blame. The bisect process is more time-consuming, requiring testing of each version, but it also requires less manual work, since git is taking care of checking out versions to test.

Finally, after finding where the problem was introduced, you should make a note of the commit identifier and run

git bisect reset

to stop the bisection process.

More `git bisect`

git bisect is not limited to finding regressions. It is a general tool to locate any change that happened because of a single commit. The "testing" phase doesn't need to involve running the program if the good/bad distinction can be made by inspecting the code. Also, when not specifically finding regressions, you can use the aliases old/new for good/bad, so that you don't get confused by the labels if, say, you're instead tracking down when a specific bug was fixed.

git blame is useful for finding when a specific line was added. But it cannot be used to find when a specific line was deleted. It occasionally happens that I notice a crucial line of code is no longer present, so I want to know when and why it was deleted. git bisect is a good tool for this: A version containing the line is good and a version not containing it is bad. With this, bisecting will find the commit that deleted the line.

There are even more features. You can record the bisect session with log and get to the same state later with replay. Editing the log in between lets you change your bad/good decisions, for instance if you made a mistake. It is also possible to use a script for the testing part with run, which makes the whole bisection process completely automatic. I rarely find a use for this, since for me the testing is typically unique to the issue, and trying to script it would often be more work. But in some cases, e.g., failing automated tests, it can be useful.

Bisect Stories

It's never as simple as in a constructed example specifically made to demonstrate how bisect works. In one of my projects, the result was often very unstable and didn't seem to find the actual problematic commit. Turned out, the compilation process didn't properly recompile everything that was needed after bisect checked out another commit. That's when I learned to run git clean -dxf between bisect steps to make sure I had a clean tree. Usually it's enough to do a regular clean build and installation, though.

Other times, it's happened that I haven't been able to find a working version, no matter how far back I go. That's usually caused by a toolchain update on my computer, or sometimes a dependency update that doesn't get rolled back when checking out earlier code. This can cause serious self-doubt when you first encounter it, but after a few times, you start recognizing it. It helps immensely if the dependency management system uses lock files, or if you specify precise versions for them (some teams ensure this by committing also the dependencies), so that the historical system behavior remains reproducible.

git bisect skip is there when some commit has other problems that make it not testable. But if you end up skipping many commits, git bisect may in the end just tell you "The problem is in one of these 10 commits". One common situation where this can happen is when a feature branch has been rebased and the main branch has changed something that new functionality on the feature branch depended on, making all the older commits on the feature branch unusable.

But after the obvious potential problems are dealt with, I find I can trust git bisect fully. One case that has stuck with me was when bisect pointed to a commit that seemed to have nothing to do with the bug. I did rerun the bisection but got the same result. After accepting that, yes, this was the commit that introduced the bug, I dug deeper and discovered that the change had in fact exposed a race condition that already existed in the code. This might have blown up in our face at any time. Being able to focus on just a small change discovered with bisect greatly reduced the time to fix that bug, and perhaps we never would have figured it out without that focus.

Introducing Git Hammer: Statistics for Git Repositories

Jaakko Kangasharju — Mon, 21 Jan 2019 08:37:19 +0000

I've been feeling that every time I'm in a longer project, I end up putting together some simple shell scripts to gather statistics from the project git repository. These were always ad-hoc, needed to be tailored to each specific project, and the only way to follow the progress of the numbers was to run them and save the output.

Recently I had some free time, so I decided to finally write a proper program to do this stuff for me. It can now do everything that my ad-hoc scripts previously (actually somewhat more even), it stores whatever it computes so it takes very little time to keep up to date, and I'm happy enough with the functionality and code that I'm comfortable releasing it. So, meet Git Hammer.

What Does It Do?

The main thing that my scripts always had was the count of lines per person. Essentially, the script would run git blame on every source file and add all the counts together. This is, in a way, the core of Git Hammer also. Another thing is to count all the tests and group those too by person. But Git Hammer knows about all the commits, so it can do many kinds of statistics based only on the commits and not their contents. At least in theory; I haven't yet implemented much.

One nice feature is support for multi-repository projects. In the project where I was working when I first began to plan Git Hammer, we had the main app, but also several support libraries in other repositories. These libraries were being developed by the same team, but they were separated to allow other projects to use them too. So it makes sense to combine all these repositories under one set of statistics.

Graphs

Let's take a look at some graphs that Git Hammer can draw. I'm using the dev.to repository for these. First, let's look at line counts per author:

Well, that certainly displays the case where existing code was imported into a new repository. It's also not a very good graph: The legend with the author names is covering part of the data, and not nearly all authors are displayed. Running this kind of program on a repository with many many contributors can definitely uncover problems.

How many tests are getting written? Let's look at just the raw test counts this time.

Looks like a nice development. New tests are being written along new code.

We can also look at when the commits are happening. There is one graph for days of the week, and another for hours of the day.

Looks like primarily of a day job: The majority of commits happen Monday to Friday, roughly during business hours.

By the way, this last graph uncovered a bug. I had been very happy with my graphs, but when I first saw the hour-of-day graph for dev.to, it was showing most activity happening in the night. Of course, this was a time zone issue: At some point in the processing, the commit times got converted to my local time zone (Berlin). Since most of the commits happen in New York, this pushed the times 6 hours ahead. So I did what seems to be the most common advice: I store the time zone associated with the commit explicitly in the database, and then use that when reading for display.

How Long Does It Take to Run?

Running git blame on every file in the repository probably sounds like it takes a long time. And it can. The main repository of my old project requires about 4 minutes. Of course, Git Hammer doesn't run this from scratch for every commit. Rather, it uses diffs provided by git to adjust its counts only where they might have changed. Processing the dev.to repository (about 1300 commits, 70000 lines of code in the latest version) took only 6 minutes on my Macbook Pro.

Larger repositories are a different case. My old project has over 33000 commits, maybe 250000 lines of code, and it takes over 12 hours to go through. Luckily, the process was using only about 20% of CPU and even towards the end well under 2 GB of memory, so I could keep working while it was running. Still, it may be that the time needed grows faster than the size of the repository, so trying a really massive repository is probably not a good idea.

Future Plans

Git Hammer is already almost a usable library. That will likely be the next step: Fix things that don't make sense in a library, maybe add some configuration points if needed, and upload to PyPI. I have also a long-term hope to make a Web service that uses Git Hammer to display project statistics on the Web.

Any contributions are welcome, starting from just ideas for features. The code base is also not very large, since a lot of the heavy lifting is handled by GitPython and SQLAlchemy. So it is probably comprehensible to many Python developers.

Caution

Deriving statistics from code is for entertainment purposes only. They have very little meaning, and none outside the specific project team, and should not be used as a basis for any decisions.

That new thing everyone is raving about? It's not so hot after all...

Jaakko Kangasharju — Tue, 15 Jan 2019 14:51:36 +0000

I'm sure you've heard of that new <best practice>? Everyone has been writing about it, saying how it's the best thing ever and you should immediately adopt it to remain relevant. Well, I've looked into it and I don't think it's that good. You probably shouldn't use it. Sure, sometimes it can be good but not always, so don't jump on the hype train without thinking.

So how can you tell when to use <best practice>? Turns out, it's quite simple. Look at your situation. Does <best practice> make sense in your situation? If it does, use it. If it doesn't, don't use it. It's that simple! Now that you have an easy way to make that decision, hopefully the hype dies down and we can adopt the good parts of <best practice>.

DEV Community: Jaakko Kangasharju

I had a bug with ForEach and Range in my SwiftUI code

Detecting Cute Animals on iOS: A Continuation

Converting the Model

Reading the Camera on iOS

Cropping the Photo

Running the Classifier

Summary

Programming with State: Concurrency

Invariants

Entities and Messages

Mitigation in Practice

Summary

Programming with State: Architectural Roles

Architecture with No Pronounceable Acronym

Overview of Roles

Roles instead of Components

Other Architectures

Scenarios

Summary

Programming with State: Values and Entities

Values and Entities

State

Dependencies

I Am Unhireable

I am in my mid-40's

I've worked in the same place for 10 years

I learned to program in the university

My Github is small and irrelevant

I don't have a "passion" for coding

I have a PhD

Nevertheless...

Tennis and Amplifying Small Differences

Basketball for Nerds

Thomas Eckert ・ Apr 10 '19

Generalization

The Case of Tennis

Appendix

Detecting Cute Animals with Machine Learning

Training and building a custom image classifier mobile app

What to Build?

Transfer Learning with Inception

Data for Training

Building the Model

Reading the Camera on Android

Making Classifications on Android

Summary

My Developer Interviewing Principles

An Interview Is Not an Exam

An Interview Is Not an Interrogation

There Should Be No One Right Answer

My Opinions Are Not the Only Correct Ones

Sometimes I Ask Bad Questions

I Don’t Care How Well You Function under Pressure

But I Do Want to See Your Code

Fixing Regressions with Git

git blame

First Attempt

Look Only at Actual Changes

Go Further Back in History

git bisect

Setting up Bisection

Evaluating Commits

More git bisect

Bisect Stories

Introducing Git Hammer: Statistics for Git Repositories

What Does It Do?

Graphs

How Long Does It Take to Run?

Future Plans

Caution

That new thing everyone is raving about? It's not so hot after all...

`git blame`

`git bisect`

More `git bisect`