Chatbot 3: Long Term Memory using a file

In the previous post, we covered Golang Slices used for short term memory for the chatbot. As we noted – this only lasts as long as the program is running. Now, let’s see if we can keep information when the chatbot goes offline. That is – long term memory. There are a few different approaches for long term memory. Writing to a file or to a database are the most common. For this example, we’re going to use a file. For future development, most use a database. This allows the database to handle the i/o (input/output) and maintain performance. This is because a database is designed specifically for the use case of storing and retrieving data.

Step 1: Breaking it down

Conceptually, File I/O is actually much more straightforward then previous concepts. The program needs a destination file, a payload, and permission when writing to a file. It needs a source file, destination and permission when reading from a file. It will also need to know how to interpret or parse the data when reading. Thus, it also needs to store the payload in a consistent format. There are a large number of guides in any given programming language for writing to / reading from a file. I’m not going to recommend one currently – and will focus on the actual implementation.

Step 2: File I/O in golang

The first example we have, is creating a file. Here is how to create ‘test.txt’ and avoid a memory leak. A memory leak is when something is reference by the program, but the code flow of the program no longer has context for that reference.

// Get a read/write file reference by creating it if it doesn't exist, or appending to it if it does exist.
file, err := os.OpenFile("test.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666) // This provides a reference to the file 'f', or an error if it cannot. 

// check for the error. 
if err != nil { // a 'nil' error means no error occurred. 
  fmt.Println(err) // print the error
  return // stop processing the context.
}

// ##IMPORTANT## Always close the file reference to prevent a memory leak!
defer func() { // defer is an instruction that runs when the 'context' closes.
	_ = file.Close() // close the reference!, the underscore is a way to ignore the value returned by Close()
}()

Reading from an open file reference

// Reading each line in a file into an array
var lines []string // declare the slice to hold the data
scanner := bufio.NewScanner(file) // use the buffer input output package to start a file scanner
for scanner.Scan() { // loop through the file scanner
  lines = append(lines, scanner.Text()) // add the scanned line to the lines slice.
}

Writing to an open file reference

text := "Coming to a file near you" // string to write to file
_, err = fmt.Fprintln(file, text) // write the data to the file
if err != nil { // error handling if file I/O fails.
   fmt.Println(err)
}

\\ or alternatively
linesWritten, err := f.WriteString(text)

Step 3: Date storage format

There are many ways data can be stored within a file. If you’ve messed with files in the past – you’ll know a bit about text .txt files. Maybe you’ve seen markdown .md files. There are many different types of files – and these all serve different purposes. markdown, text, and even .csv files are all Human Readable Formats. If you open any of these in a text editor, you’ll be able to make sense of them quickly. However – they aren’t exactly the best for storing data from a program. For this particular example, we are going to simply use a txt file to place human readable data into the file.

Step 4: Chatbot Enhanced Memory

  1. launch program
  2. check if history exists (create it if it doesn’t exist)
    1. load the history into the history slice
  3. send a friendly greeting and store it to history
  4. send a simple instruction and store it to history
  5. Have the user put something in and store it to history
  6. echo what the user put in and store it to history
  7. Check if the user typed ‘history’
    1. Store the request to history
    2. don’t repeat the history in the history file, just count the number of lines.
    3. Loop through the history slice
    4. print the key and the value
  8. repeat 4-7 until the user says ‘bye’

Step 5: Putting it all together

Now that we’ve covered the details, we can run the program. We’ll find that it isn’t much different from running the program during lesson 2. However – when we say ‘bye’ and start the program again, that’s where the changes are noticeable.

  1. run the program
  2. type hello
  3. type history
  4. type bye
  5. run the program again
  6. type history

Example output

Here is the final example used above. It is also available in github in chatbot, lesson 3

func main() {
	f, err := os.OpenFile("history.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)

	// check for the error.
	if err != nil { // a 'nil' error means no error occurred.
		fmt.Println(err) // print the error
		return           // stop processing the context.
	}

	defer func() {
		_ = f.Close()
	}()

	// string for storing input
	var input string

	// Slice for storing the history of the world.
	var history []string
	scanner := bufio.NewScanner(f) // use the buffer input output package to start a file scanner
	for scanner.Scan() {           // loop through the file scanner
		history = append(history, scanner.Text()) // add the scanned line to the lines slice.
	}

	fmt.Println("Hello, World!")
	fmt.Println("I am Echo! Please tell me something to say by typing it in, and pressing enter!")
	_, _ = f.WriteString("Hello, World! \n")
	_, _ = f.WriteString("I am Echo! Please tell me something to say by typing it in, and pressing enter! \n")
	// Stop when we see "bye"
	for input != "bye" {

		fmt.Print("You: ")
		scanner := bufio.NewScanner(os.Stdin)
		for scanner.Scan() {
			input = scanner.Text()
			break
		}
		_, _ = f.WriteString("You: " + input + "\n")
		// Add the new input to the history
		history = append(history, input)
		fmt.Print("Echo: " + input + "\n")
		_, _ = f.WriteString("Echo: " + input + "\n")
		// Check if the input was the string 'history'
		if input == "history" {
			// We won't add the history to the history, instead we can add how many lines of history exist.
			_, _ = f.WriteString("<Truncated> " + strconv.Itoa(len(history)) + " Lines of history repetition.")
			// iterate (loop) through the history
			for k, v := range history {
				// print out the Key and the Value
				fmt.Println(k, v)
			}
		}
	}
}

I don’t know about you – but that code is starting to look like my pantry after my kids try finding the peanut butter. In our next lesson – we will start to look at organizing code a little bit better.

Chatbot 2: Improve the chatbot with a memory

Many people can recount their first memory. We don’t really know how we do it, or why it is a first memory. It’s just there. We can also remember things more recently – or – remember information that is imaginary. For this lesson, we will be covering memory in terms of programming. Because a chatbot without a history, is like playing Memory and just choosing random cards each turn. Lesson examples can be found on Github

Step 1: Understanding the Lesson

Before we dig into code – let’s see if we can understand a little more about storing things in program memory. For most languages a developer can write, there are a few different ways to store data. In previous lessons, we’ve covered what a variable is – including a few types of variables. In this lesson, we are diving more deeply into the array type of variable.

One way to understand an array, is to think of series of mailboxes. They all have a number on them so the delivery person can put the mail into the correct slot. When a large neighborhood is built, there are only so many houses – and – there are only so many mailboxes. Depending on how this neighborhood is setup, any new houses will either be an easy addition (or) for a grouped mailbox setup – a new group would eventually need to be created. Arrays are like the grouped mailbox setup.

But what is an array, really? An array is a declared list of keys and values. From the mailbox example, the mailbox number is the key, and the things inside the mailbox are the value. At a more physical layer – arrays are blocks of ram reserved for storing sets of specific types of variables. In different programming languages, there are different mechanics for using arrays. In c++, one needs to program what happens if you try to add more keys to an array then the original requested amount. In golang – the language itself handles this for you using slices – so you don’t have to program the tedious memory management. Most modern languages handle the array management operations in some similar fashion.

Step 2: Using an Array in Golang

Now that we’ve briefly covered the definition of an array, let’s look at a few array operations. The Golang Tour provides an example of arrays here: A Tour of Go (golang.org).

Example from the golang tour with notes:

func main() {
  // creating a new array
  var a [2]string

  // adding to an array using a key
  a[0] = "Hello"
  a[1] = "World"

  // looking up something in an array
  fmt.Println(a[0], a[1]) // Prints Hello World

  // Creating an array with values
  primes := [6]int{2, 3, 5, 7, 11, 13}
  fmt.Println(primes) // Print [2 3 5 7 11 13]

  // removing the item with key 3 from the array
  var slice []int                                // create a 
  slice - notice how it doesn't have a length specified?
  slice = append(slice, 1, 2, 3, 4, 5, 6) // add values 
  to the array
  removalKey := 3                                       // set a key for removal

  // remove an item by adding the first portion (before the key, and the second half after the key together) 
  slice = append(slice[:removalKey], 
  slice[removalKey+1:]...)
  fmt.Println(slice) // Print [1 2 3 5 6]        // notice how the '4' is gone?

  // iterate (loop) through the slice and print the key and the value
  for key, value := range slice {
    fmt.Println(k, v)
  } 
}

Step 3: Chatbot wants a slice

With only a few lines of change, we are going to have chatbot keep a short term memory of the conversation. Here is a breakdown of the logic we’re going to use for chatbot:

  1. launch program
  2. send a friendly greeting
  3. send a simple instruction
  4. Have the user put something in
  5. echo what the user put in
  6. Check if the user typed ‘history’
    1. Loop through the history slice
    2. print the key and the value
  7. repeat 4-6 until the user says ‘bye’

To start, we will initialize a new slice.

// Slice for storing the history of the world.
var history []string

Next, whenever we get new input, we are going to store it in the history

// Add the new input to the history
history = append(history, input)

Finally, chatbot is going to check for a ‘prompt’ that it will use to print the history.

// Check if the input was the string 'history'
if input == "history" {
	// iterate (loop) through the history
	for k, v := range history {
		// print out the Key and the Value
		fmt.Println(k, v)
	}
}

Step 4: Putting it all together

The completed above code can be found in this github repository. There is a main.go file (the code) and a README.md file (the directions).

To get the following output, run the program, and enter the following lines

  1. type test
  2. type cool
  3. type history
Program Output

Well, we’ve now got a chatbot that is still a copycat. It can now recount all the things it has copied. This ‘history’ only lasts while the program is running. In future lessons, we can explore more permanent memory, command logic, and making the code organized.

Chatbot 1: Build a simple text bot

Wouldn’t it be awesome to build a chatbot like those hackers we read about in the news and see on movies? This is the beginning of that journey. If you’ve read the previous post, you might remember it recommended downloading Golang, and, getting an IDE or text editor. I recommend trying out Goland. If you’ve not installed these – please do so now. There are a number of ‘guides’ for getting started in golang on the internet – These can be a great cross reference point – or – even better – a Hello World tutorial. (For some reason developers like to try out languages by writing a program that says ‘Hello World’). Step through the tutorial – it won’t teach a lot – but it gives a basic, very basic, program – to make sure you’re all setup correctly.

Step 1: Hello World

I recommend following: Hello World | Learn Go Programming (golangr.com)

Step 2: Explanation

Golang has a few fun ‘pieces’ to it. Here is the example code from the link above:

package main

import "fmt"

func main() {
   fmt.Println("Hello, World!")
}

In the above – it has a couple of specific items. package main identifies what this piece of the program is. In Golang, main is the primary piece – the place were everything starts. When golang compiles (takes the code and turns it into machine code) it looks for main as the start point.

The next line, import “fmt” indicates that golang needs to go grab someone else’s code. A thing called “fmt” which is a format library for strings. If you’ll remember from the last post, strings are a type of variable. Now, you might have guessed this from reading it, but, the fmt package has some prebuilt items that make golang quite powerful.

The next line func main() { is the beginning of a golang function. A function being a collection of commands that run when that function is called. You can see that it has an opening curly brace – and – that there is a closing curly brace further down the screen. These braces indicate where the function starts and stops. In golang – the word ‘main’ continues to be important. The main package that we noted above? Well, the compiler looks for a main func inside the main package. It uses this when the program starts.

The final piece, is the actual command inside the curly braces. This is what tells the computer something needs to be printed to the screen. It uses the fmt library to print, because printing is such a common operation in programming, it’s best that everyone not reinvent the wheel printing out things.

Step 3: Echobot

Now that we’ve got a working helloworld program, let’s see if we can turn it into an echobot. In certain languages, echo is the word used to describe writing a statement to a terminal window. It’s not universal, but it seems fitting for a bot name. Now, to start of coding this bot, let’s think in terms of flow (and remember, this is just a text bot – nothing too fancy.

  1. launch program
  2. send a friendly greeting
  3. send a simple instruction
  4. Have the user put something in
  5. echo what the user put in
  6. repeat 4 and 5 until the user says ‘bye’

Well, in looking at this – the first 3 steps we know how to do. The 4th will require knowing how to get something from a user. 6 is the most complicated. We need to keep getting things, and, we need to check if the user is done.

package main

import (
	"bufio"
	"fmt"
	"os"
)

// main function
func main() {
	var input string

	fmt.Println("Hello, World!")
	fmt.Println("I am Echo! Type something, and pressing enter!")

	// Stop when we see "bye"
	for input != "bye" {

		fmt.Print("You: ")
		scanner := bufio.NewScanner(os.Stdin)
		for scanner.Scan() {
			input = scanner.Text()
			break
		}

		fmt.Print("Echo: " + input + "\n")
	}
}

The example above introduces the golang for loop. We can cover that more in a later lesson. For now – just understand, for will keep running until the condition to the right is satisfied. We’ve also introduced a bufio.NewScanner(os.Stdin). You don’t have to worry too much about what that means right now, just that the whole string, means “read what the user typed, and then store it in scanner”. The next for scanner.Scan() Is telling the program to actually wait until the user completes what they are typing. Computers like to be fast when they can. They aren’t doing anything when waiting for a user to type. In golang, it is more obvious execution is being blocked by …*cough* the user *cough*.

Running the above program, and typing in test<enter>cool<enter>bye<enter> will have something like the following output:

example program output screenshot.

There are lots of things to explore in building a chatbot – and in building more advanced code. We’ve not even begun to scratch the surface yet.

Introduction to Programming 000

Quite some time ago, I went to college for Computer Science degree. I knew that I wanted to provide for a family, that I didn’t want to perform a job with a lot of manual labor, and that doctors and lawyers would fit the bill. But I also knew that I was good with computers, I’d hacked a few games after all. I’d even written a few programs. In fact, the first one I ever “wrote” I can still provide from memory.

10 print Hello World
20 goto 10

But let me get into what this post is really about. An introduction to the world of programming. Currently, I’m teaching my son 6th grade math. It’s introducing him to Algebra. Algebra is cool. It makes math useful, repeatable even. It’s kind of what programming does. The first things you learn in school are definitions, terminology, the words, and what they mean. That way you can communicate with everyone else that learns about the same subject. Without further ado, let me introduce you to some common concepts of programming.

Operators

Just like 6th grade math, you need operators to do anything. Consider 2+2. It equals 4. That’s what your 1st grade teacher taught you. The ‘+‘ sign is the most important piece here. It is the operator. It tells you what to do with the things around it. There are lots of operators out there. Most of the ones you learned in Math apply. And there are few other special operators – but we need to cover other items before we introduce them.

Variables

Leaning back on algebra, variables are representational. They can vary in what they represent. For example, if we say x is a variable, and we have an equation x+1=2, we know that the x variable is 1. Or, if we say, the total is x+1, we can set x to a certain number, say 2, and know that the total is 3. Variables are a huge part of programming. Without them, we wouldn’t be able to record the result of operations. Now, speaking of operations, one of the special operators, is the assignment operator.

In many languages, the assignment operator is notated by the = sign. In other languages, it is a combined => symbol. An example of an assignment operation, would be as follow.

x = 10+2

if we were to print the value of x, it would say 12.

Types

Unlike Algebra, programming variables can be different types. This is because, different things need to be tracked in a computer. In algebra, you’re just tracking numbers. Examples of different types are:

  • integer
  • boolean
  • string
  • object
  • float

An analogy would be something like measurements. If you try to measure flour for a cookie recipe, and you use 2 tablespoons instead of cups – or – you used 2 grams instead of 2 cups – you end up with goop (I know from experience, goop doesn’t taste good). And for this reason, programming languages use types. Keeping track of a of number is an integer. Keeping track of someone’s name, well, that’s a string, and keeping track of your next door neighbor and their email address (is creepy) but that group of types, that’s an object.

Differences between Programming languages

I’m not going to cover a lot on different programming languages at this time. Just know they are out there. A lot of the core principles are the same, or similar, even if the terminology is different. This is because the C programming language became an effective way for programmers to convey ideas. If you’re into language history, think of it as the ‘Latin’ of the programming world in that a lot of other languages stem from it.

Next step

At this point, I’m going to recommend downloading Golang and finding a text editor or IDE. Because eventually, we’ll get started on using some of these concepts in an actual application. Once you’ve gotten an IDE – head on over to Program 1: Build a simple text bot!

Imposter Syndrome (Opinion)

I had not heard the term Imposter Syndrome until recently. It’s an interesting concept – feeling like a fraud, even, and especially, while being successful at something. Apparently it is prevalent in the Computer Science degree field. I’m probably one of the most ambitious people I know – and I’ve never struggled with imposter syndrome. Overconfidence? yes. Feeling like a fraud? no. I’ve recently seen just how exposed new developers are to the concept o Imposter Syndrome – and how much it seems to affect them. I had another name for it back in the day. Self Doubt, or, Lack of Confidence. I’m going to dig into this a little bit more through this post.

I’m not psychologist – I’m just an engineer that has read too many blog posts and watched people struggle and want to help where I can. I think a large amount of the imposter syndrome is driven by how we learn about programming. Many of us learn programming by way of failure. We are given an extremely cut and dried problem – and told to solve it. Or we are given a clean working environment an told how to build the ideal solution. Real world programming doesn’t fit nicely into a box in many scenarios. (I can’t speak for Boeing, SpaceX, Tesla, or other high requirement threshold developers – they have different tolerance levels). Anyhow – this sandbox we learn in, doesn’t exactly prepare us for real world scenarios psychologically. It’s not like woodworking – the results are tangible. In C.S. it’s possible to make something good, an ship it. In woodworking – the angles have to be perfect – or else.

The self confidence is further eroded by the mistakes in execution. I was ahead of most of my class turning things in to professors. Why? Because I didn’t know if I would have time to work on it later. I had two jobs in college – and was still living with my parents. I couldn’t guarantee I would be able to study up until the deadline. I only had now to do the homework. Maybe I have A.D.D. – I never had trouble with object permanence though. I digress.

My first programming job was almost two years after graduation. There weren’t any entry level jobs where I was located in ’08-’09. Everyone wanted experience. I finally landed that job. A dream job. At a video game company. It was great – then my personal life took a hit – and I was laid off. I did a ton of soul searching, and job application at the same time. I was only out of a job for 20 days. I survived. But I had a ton of self doubt. I never felt like a fraud though – just that I wasn’t good enough. Or if I didn’t do 110% I’d get laid off again. I was also ambitious to a fault – and trying to move up the corporate ladder… I used my ambition to overcome self doubt.

The number one destroyer of confidence for developers isn’t hard problems. It isn’t difficult projects. It’s bug duty. Stick a single developer in a triage roll, and you’ll burn them out. Fast. Developers like a good mix of creating new things. I’m an oddball – I like troubleshooting things – they are like puzzles to me. Creating new stuff – eh – to a certain extent. But I don’t live for it. Re-creating something, that much more fun. I have a benchmark – and I can improve upon it.

This is probably the most rambling post I’ve written to date. Imposter Syndrome is something I’ve seen first hand. I had a very skilled developer. Possibly a speed demon. But he had some very specific imposter syndrome. It slowed him down, and, and it he ended up relying heavily on processes, and staying within the constraints of processes to justify himself as a developer. Process is great for judging one’s self. It provides a measurement. Unfortunately, sometimes the process isn’t sufficient – or isn’t malleable. This can lead to a slow down in an otherwise performant individual.

Another imposter syndrome symptom became evident to me, as I worked with a developer that wrote good solid code, and often had very in depth analysis of ongoing team politics, but had trouble speaking out in turn. The self confidence became apparent in 1 on 1’s.

Some of my conclusions I’ll draw around Imposter Syndrome, is that each type of developer represents their imposter syndrome differently. Speed Demon‘s try to write more code, Maintenance Master‘s move slower, Seeker’s look deeper – and Emergency Response Hero‘s just don’t respond. If you are struggling with imposter syndrome – maybe try to take some time retrospectively to figure out what kind of developer you are – and reach out and talk to someone. Leave a comment or send a response – IWBTD is intended to be a resource to help developers – especially those early in their career.

Dev Profile: Seeker

I’ve only encountered a few seekers in my professional career. That said, they stick out like a soar thumb. They are capable of delaying products, finding bugs in perfect code – writing code that looks incredible, but, might miss something obvious. They dig deep into the problem when troubleshooting – and heavily blame others and existing code for any problems. It’s amazing the amount of technical detail they can find within a code base – I’d even consider them genius level in certain areas, but that said, let me cover some strengths and weaknesses and areas of growth – if you find you are a seeker.

Strengths

Seekers are passionate. It’s a huge asset. They really care about doing things properly, about digging into problems, and providing solutions. They care about their code. Whatever their motivation is, they strive for perfection. If they find they’ve landed in a working culture where that perfection isn’t appreciated, or if they perceive that others are not as focused on doing things properly. They’ll speak out about it. And if they stop speaking out about it – they’re considering leaving. The passion is a huge asset in any dev culture.

Their passion also drives them to know intimate details of how a programming language works. I’m a passionate developer, but this has not been my strength in the past. I know random things – more like trivia – I’d do well at programmer jeopardy. But a seeker – they know the intimate minutia of edge cases. You won’t be able to prove them wrong. Not on a specific detail. This a huge strength. They know things – and if they don’t know it, they figure it out. They are a valuable resource to others and often end up in SME type positions.

Another attribute that is a strength is their strong work ethic. I’ve not found a seeker that isn’t consistent in their work. They really emphasize their work – they are driven and often ambitious. They’ll often have side projects – and those side projects often dovetail with what they are working on professionally. These side projects can sometimes be a distraction as well. Especially in more junior level seekers.

Weaknesses

Seekers are incredibly prone to getting lost in a code base. The larger the code base, the more likely to get lost. The term can’t see the forest through the trees is the most relevant. Sometimes, seeing the forest is needed to solve the problem.

Making a mistake is a huge detriment to seekers. They aren’t able to easily brush off failure. One incident I remember in particular, was a bug that made it to production. It was a benign bug – a misunderstanding in requirements. But it really bothered the developer. They had recently started a new project, but wanted to drop everything to understand the bug they’d written. They even went so far as to work on the bug in the free personal time. Unfortunately, it undercut there progress on the new project.

Being correct to a fault. I’ve argued with seekers on a few occasions. Not because they were wrong, but because their focus was off. They knew exactly what they were talking about. They knew their context. But they had a really hard time adjusting their viewpoint, because they knew they were right. The blind adherence to their understanding is possibly the biggest weakness early in a seeker’s career.

Sometimes, what they know causes them to be judgmental. Knowing the proper way to do a thing can make it hard to accept a shortcut – or a workaround. This in particular can cause delays – or even destructive implementations of work-arounds. I’ve seen this on a number of occasions. It feels like sabotage when you’re managing a developer that does this. For a seeker – implementing the work-around is completely against their nature. Getting their buy-in, their agreement, is difficult and crucial in this scenario. And don’t be surprised if they point to the work-around as the source of problems anytime in the future. Especially in meetings with others. They’ll crusade to ‘fix’ the work-around whenever they have a chance.

Areas for Growth

Find the forest. Understand that there are often larger things afoot then a single developer can handle. Larger often then a single manager can handle. Understand there is a forest – and sometimes, trees grow crooked to reach the light. Growing a perfect tree is great for a back-yard. But growing a perfect tree in an imperfect forest isn’t always possible. That is to say – larger projects aren’t always as clean cut as pet projects. Expect great code – but don’t get too bent out of shape when great code isn’t perfect.

Understand that perfect is the enemy of good. Strive for perfection, but understand what is good. Know when something is ready to be shipped and supported – instead of focusing on winning Nobel peace prize for the perfect for loop execution.

Guide others through knowledge sharing. Don’t bludgeon other developers into your way of thinking. You’ll alienate them – and they’ll remove your code the first chance they get – because they didn’t understand, and, they didn’t get along. Teach others about some of the intricacies. But remember – sometimes, those details don’t matter. Know when to back-off, when to listen, and when to let go.

Do you think you’re a seeker – or is there one you know? Maybe you have a different idea for a seeker? Please comment and let me know!

Dev Profile: Speed Demon

Miriam-Webster defines a Speed Demon as someone that moves or works very fast. Some people just have this need for speed. Software developers at not immune. There aren’t many speed demons out there. Turns out, a lot of risk takers don’t like the idea of sitting behind a desk pecking away at a keyboard. But for those that do exist, here are some more common traits, and some things to look out for.

Traits

Has Fast task completion

Give a speed demon a well defined ticket, and the turn around time will be significantly less then their counterparts. For some reason – racing through tickets to the solution is was they feel gives them the edge.

Missing attention to detail

However, watch out for simple mistakes. It’s not laziness – it’s just slow proofreading code over and over again. That’s what tests are for. That’s what code reviews are for. It’s great that they can do things quickly – but a little bit of slowdown, and fewer issues is a much stronger result.

Has a strong grasp of the whole code base

The speed demon truly tries to grock the entire code base. It’s the only way they can move quickly. Given a task, they’ll study it upfront, make assumptions, try some things, then come back with a working – although not always optimal solution. It will work though. Mostly.

Missing blindspots

A speed demon’s true weakness is blindspots. If they miss something, and moving fast they are going to, they truly miss it. The earlier in the process it is, the bigger the mistake. Usually it’s a problem understanding, or missing, requirements. Sometimes it’s just forgetting to dot the i’s and cross the t’s.

Has a tendancy to showoff and brag

Most of the time it is a good thing for a developer to humbly celebrate their victories. Some speed demons aren’t humble – they try to offset their speed with victories. They won’t mention the mistakes, although they often reflect heavily upon them.

Missing compassion when others are slow

From a soft skills perspective – other people being slow is annoying. And expressing that isn’t a good soft skill – it’s more a lack thereof. Understanding what a slower more consistent develop brings to the table is a huge boon to the speed demon.

How to get ahead as a speed demon?

Try slowing down a little and making less mistakes. Try to lift up the team, instead of pushing / pulling them in the direction you want to go. Beware of scrum – because the points are based on the team’s velocity – you won’t fit well with the smallish tickets created on a scrum team. That said, you can use it to your advantage and plow through more points then others on the team. If your manager is attentive – and you aren’t making a lot of mistakes that everyone else has to clean up – you’ll do quite well in scrum.

How to work with a speed demon?

Mass migrations, large refactors – these are their bread and butter. Everyone else might think they are tedious, but for someone trying to fly through something quickly – these work well for them. Make sure they have the project in hand, maybe have them test the base case, then move on to the remainder of the task. Don’t make the same code review expectations on a project like this. In fact – make sure the base case is solid – if it isn’t, don’t let them continue until they can make a migration successfully.

Try to keep them from overrunning projects with unique properties. In their zeal for accomplishment they might miss the larger picture, and possibly block up the entire team with their CR’s that aren’t quite what was needed – but are super close.

Think you are a speed demon, or know one? I’d be curious to know! Feel free to leave a comment. If you like what you read, please share it! – and, as always – thank-you for reading!

5 Interview tips for the Developer Culture Fit

I’ve seen a lot of articles about missing out on jobs because someone wasn’t a culture fit. Older developers think it is because of age discrimination. Younger developers think they don’t have enough experience. Minorities blame it on being a minority, and the same for special interest groups, and don’t forget Gender discrimination. Naturally, that’s what’s at fault. Well, except for a few scenarios… I call hogwash! Poppycock. Etc. You didn’t get the job, because you weren’t a good fit. But there is hope. Here are 5 ways to understanding culture fit and land a job where you’ll fit right in (pun intended).

Soft Skills

Depending on where you are in your career, you may or may not have heard of soft skills. Soft skills are a collection of empathy related abilities tied to Emotional Intelligence.  I cannot emphasize enough how important soft skills are to get ahead. I’ve seen incredibly talented individuals get stuck in their career, or even fail completely at a company because they couldn’t relate to the people around them, or their direct supervisor.

Take myself, for example, I had a huge problem with being passive aggressive. I didn’t know what it was until someone sarcastically called me out on it. I had to go lookup what it meant, and then I apologized to the individual. As I became more aware of the problem in myself, I was much more easily able to recognize it in others. It spurred a desire to understand my coworkers better – because I had come to realize the importance of teamwork, and how to maximize the teamwork.  Now – I’m by no means perfect – and I’m still learning more about this all the time.

If you are not familiar with soft skills, take some time to read through some quick posts online – or find a self help book. I can guarantee it will improve your ability to tell if you’re a good culture fit before your interviewer! Which in turn will enable you to present yourself as a good culture fit. Or decide you didn’t want that job anyhow.

Quit Complaining!

Nobody likes a quitter complainer! Ever had that coworker that just couldn’t catch a break, or that linked in contact that could never land that job they were after, or the friend that just didn’t like their situation? Well, even if you haven’t, you’ve probably heard complaining, or participated in it yourself. Complaining is annoying! Complaining, even on a public form, can work against you in the hiring process. Over 70% of people I’ve interviewed I needed to reject, because they were used to complaining about a bad manager, team, or coworker – they couldn’t focus enough to identify what they could do differently. 

If you’re guilty of too many complaints – try to take some time and focus on yourself as the problem in the situations.  Maybe, there’s more to the situation then you realized.  If anything, reducing the amount you complain everyday will improve your interview skills. 

Introspection

The concept of introspection has been around for a long time. See Know Thyself . Just because the concept is old, doesn’t mean that it is easy. And in the concept of a developer interview – it means something more entirely. I’ll use a couple of examples here to make this a little more obvious, if not exaggerated.

Let’s start with Gary. Gary is fresh out of college, and has been applying at startups and large companies but can’t seem to land the job. He’s never had an internship, but was one of the top achievers in his university. He’s extremely confident he can solve any problem or fill any roll given the opportunity. Gary flies through interviews, he’s got no problem with the technical challenges – and the questions he’s asked don’t seem to be a problem. He can’t figure out why they told him no. What do you think might be his problem?

Larry is another interviewee.  He’s much more senior then Gary, he’s had a good number of positions as different companies over the years, but now, he feels older and has grown frustrated as jobs that were once easy for him to land now turn him down on culture fit of all things. A typical interview goes pretty much how he remembers them all going. Except now, he’s able to fill in the blanks of how exactly he’d solve that problem he saw a few years back. Later he finds out he wasn’t a good culture fit. Why would Larry be having a problem getting a role he’s clearly qualified for?

Going back to Gary, A problem I’ve seen in myself, and about 10% of candidates I’ve interviewed (disclaimer: I don’t interview junior roles frequently). Was that I had a brilliant candidate that couldn’t figure out where they’d made a mistake. Asking them to describe projects, or projects with team members – they had no fault in anything they didn’t.  Gary didn’t understand what he was doing wrong. He hadn’t learned to be introspective.

Larry, on the other hand, had a significant amount of introspection. HE knew when he’d made mistakes in the past, and could draw on his wealth of knowledge to solve problems, especially using them for questions in interviews.  What Larry failed to recognize in himself, is that he was getting older – yet he was interviewing like he was fresh out of college, or worse – had a complaint mindset. A younger interviewer is often naïve – willing to work excessive hours – more likely to argue from passion then experience – and attuned to learn new things.  Whereas someone with a few years experience is much more likely to argue from a position of experience, and less attuned to learn new things, when after all, the previous way of doing things worked just fine. The best tip for Larry, would be to understand himself as an experienced individual, own that, and then be passionate about new things. Show an online course, a Github repo, or something else in that manner.

There’s a lot to cover on introspection – and I’m no expert – I probably even offended someone with the two examples and my conclusions. Hopefully I didn’t.

Be Politely Passionate

90% of interviewees I’ve had – I’ve known within 2 minutes if the person was passionate. If they weren’t – that meant I wasn’t hiring them. Naturally, the interview would continue, and I would try to prod passion from the interviewee, coax it out even.  In that remaining 10%, I’d say only 1% of them actually proved me wrong.

On top of being passionate, be polite. If you’re passionate to the point where you are interrupting the interviewer, maybe you need to check yourself, that’s right – use some introspection – and some soft skills, and be passionate – but not at the expense of the interviewer.

Ask Questions

One of the worst interviews I gave, was where the interviewee had no questions for me. They didn’t even try to think of any. To me, it came of as if they didn’t care. Needless to say, they would not have been a good culture fit. It’s hard to be a passionate candidate if you don’t have any questions – stall for time if you need to – say something like, “Well, you covered a lot of information about the job – could you tell me more about the commute?”.  Just asking a simple question like that shows how much you’ve been paying attention – and increases your engagement level. 

Summary

Remember – if you’re at the point where your actually talking to someone – be it face to face or over the wire – you are actually talking to someone. That person is getting a first impression of you. That impressions will make or break your interview process.

If you liked this article, please let me know by re-sharing or commenting! Any questions/comments/not-spam interactions are appreciated!

Guide: Build a RESTful API microservice with PHP

Too many times I’ve seen people lost in the weeds arguing about endpoints, design patterns, reusability, etc. It turns out – with a decent microservice – the architecture lends itself to simplicity. For this guide – I’ll take you step by step on how to standup a minimal Restful API Microservice using PHP. I’ll be using the Waryway Microserver Engine.

Setup the Project

This guide won’t cover adding your project to GIT.

  • Create your project directory
  • Add a composer.json file
  • Add the dependencies
    • waryway/micro-service-engine
    • waryway/php-traits-library
  • Add the autoloader to the composer.json
  • Add the src directory
"autoload": {
  "psr-4": {
    "OrgName\\ImagerServerRest\\": "src/"
  }
}

At this point, your project is going to look something like this:

ImageServerRest
  src
    ...
  vendor
    ...
  composer.json
  composer.lock

Create an initial endpoint

For the initial endpoing, I’m using the example from the readme.md.

  1. Copy the Router.php from example
  2. Replace Waryway\example with OrgName\ImageServerRest
  3. Change the setRoute to only ‘/’ and respond with ‘Hello World
    • $this->setRoute(['GET', 'POST', 'PUT', 'DELETE'], '/', [__CLASS__,'helloWorld']);
    • This was mostly just to provide a ‘test’ endpoint.
  4. Run composer update
  5. Run vendor\bin\server OrgName\ImageServerRest 0.0.0.0:99
  6. In a browser, navigate to localhost:99
Tada! You got a ‘Hello World’ response.

Add a real endpoint

The real cool thing. Adding an endpoint. If you ever want to add an endpoint, make certain to use a proper router. Frameworks have them. I like nikic’s fastrouter in PHP, or Mux is also fine in Golang.

Not getting too fancy – I just add a new route (/images), and a route handler (getImageStatistics) – and set them up to return according to what I find out. The example itself gets rather long – so I’ll simply link to it in github. Once I’ve gotten all the changes in place, I can run vendor\bin\server.bat orgname\ImageServerRest 0.0.0.0:99 and try out the URL. Eureka! http://127.0.0.1:99/images now returns a list of information about the available images.

Make it Conventional

It’s one thing to build an endpoint. It’s another to publish an endpoint. What I’m saying is this: If all lightbulbs had different size sockets, it would be much harder to go to the store to buy lightbulbs. In the same way – if all RESTful api’s have a similar standard for definition to clients, they become much simpler to use overall. The OpenAPI Specification provides just that, and Zircote/Swagger-PHP is a great tool to implement it.

Start by updating the composer.json with the swagger-php composer setting as a dev require. Then run composer update. Who knows, maybe Waryway will add it to the microservice engine at some point.

"require-dev": {
  "zircote/swagger-php": "dev-master"
},

Using the documentation from Zircote is pretty straightforward and the annotations are pretty obvious. That said – I ended up using this command to generate a nice little swagger.json file:

vendor\bin\openapi.bat --pattern "*.php" --output "./src/swagger.json" --format json ./src

The easiest way to see if the generated file works correctly, is to use the swagger editor, and paste the file contents into it. The editor will automatically convert any json to yaml – and will nicely update the generate API documentation. It’s really neat!

example of editor.swagger.io

Another cool trick with swagger, is the simplicity of adding the swagger view into the codebase itself, and exposing the entire utility via an endpoint. For this guide, I’m simply adding the swagger doc to view it at /swagger.json

Wrap the Package

If you thought containers weren’t going to make an appearance while talking about microservices, you should have paid for insurance. If you are considering the cardboard boxes, plastic totes, or even shipping containers – you might want to brush up on Docker!

The first piece of wrapping a package, is to take a step back from the /src directory and create a /build directory. The goal here, is that the build directory never actual lives where the program is deployed. Once you’ve added the build directory – let’s add a Dockerfile. Now, it doesn’t have to be a fancy Dockerfile – at least not for getting started. I am going to use the 7.3.10-cli-alpine from Docker Hub as a base image.

Note: The only files you really want to work with are your composer.json, composer.lock, and src directory.

There are many different flavors of docker files. For this one, I’m not really optimizing for size. I don’t clear the composer cache, for example. Notice however, that the swagger.json generator is included in the build. I also added the build command I used as a comment.

FROM php:7.3.10-cli-alpine

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/bin --filename=composer
COPY ./composer.json ./composer.json
COPY ./composer.lock ./composer.lock
RUN composer install
COPY ./src ./src
RUN composer dump-autoload
RUN mkdir -p /assets/images
RUN ./vendor/bin/openapi --pattern "*.php" --output "./src/swagger.json" --format json ./src
EXPOSE 80

CMD php /vendor/bin/server orgname\\ImageServerRest 0.0.0.0:80

# docker build -t orgname/image-server:v1 -f build\Dockerfile .

Once you’ve gotten the Dockerfile built – simply kick it off, and remember to mount the ‘images’, or else there will be none.

docker run -it -p 80:80 -v C:/development/ImageServerRest/assets/images:/assets/images --name test orgname/image-server:v1

What now?

Well, now we’ve built a microservice, put it in a container, defined it with swagger, and even got it up and running locally – the next logical step is to ship the container! You’ll need to harden it (because containers are best not run as root). You’ll need to get it to a registry (dockerhub is my favorite currently). CICD for the entire project would be a huge boon… And, of course! you’ll probably want to write a different endpoint then my contrived ‘image stats’ endpoint.

I hope you’ve enjoyed this guide. If you have any problems following it – feel free to comment and ask about it! If there is a bug in the code – a new issue on GitHub would be greatly appreciated. If you’d like more guides like this: like, subscribe, and comment! Finally, thanks for reading!

Guide: Implementing a PhpUnit Development Environment in PhpStorm

Preperation

For this example, I will be starting with the following:

  • PhpTraits repository https://github.com/Waryway/PhpTraits
  • PhpStorm Project
    • Make certain you have a php interpreter referenced in your storm project
    • Also make certain you have Composer referenced by PHPStorm
    • A desire to test.allTheThings()
Adding the PHP interpreter reference.

Installing Libraries

First, use composer do some work. We need a copy of phpunit. Get it with this:

"require-dev":{
    "phpunit/phpunit":"6.4.3"
}

As this is a ‘library’ style app – use an exact version. The lock file gets ignored in favor of precise dependencies in the composer.json.

Now run composer update to pull in the requested phpunit package.

Setup phpunit in the general settings, under phpunit

Adding a new test

Or rather, a test skeleton

  1. Open the file to test in the Project View
  2. Place you cursor on a method to test within the file
  3. Go To Navigate -> Test
  4. Create New test
  5. I tend to set mine up with a test prefix to keep tests more obviously separate from code.
  6. Click Ok then Open up the new file
  7. Add a line to pull in the composer autoloader
  8. Clean up the Namespace reference to the TestCase if you are using PSR-4
  9. Disclaimer: Strongly avoid namespacing unit tests. They should not be built out like a code base. They are testing units.
  10. Add an empty testHydrate method.
  11. Add a super obvious assertion.
  12. Add a phpunit.xml fileI put it in the test directory.
  13. Run the test (At this point, I needed to restart phpstorm to detect that I actually had a ‘test’ to run.
  14. Build the tests out further.

Conclusion

Ending of the beginning

I’ve gone ahead and built out the ‘hydrate’ test against a trait. Note the use of the trait object that phpunit provides. Pretty slick, right?  I’ll continue to build out this repository – core libraries need the most directed test coverage.

Even adding the ‘test’ cases around the hydrate method – I have found use cases I didn’t consider while writing the code – and have adjusted the code to reflect the behavior I expect.