Tag: software development

In the previous post, we covered Golang Slices used for short term memory for the chatbot. As we noted – this only lasts as long as the program is running. Now, let’s see if we can keep information when the chatbot goes offline. That is – long term memory. There are a few different approaches for long term memory. Writing to a file or to a database are the most common. For this example, we’re going to use a file. For future development, most use a database. This allows the database to handle the i/o (input/output) and maintain performance. This is because a database is designed specifically for the use case of storing and retrieving data.

Step 1: Breaking it down

Conceptually, File I/O is actually much more straightforward then previous concepts. The program needs a destination file, a payload, and permission when writing to a file. It needs a source file, destination and permission when reading from a file. It will also need to know how to interpret or parse the data when reading. Thus, it also needs to store the payload in a consistent format. There are a large number of guides in any given programming language for writing to / reading from a file. I’m not going to recommend one currently – and will focus on the actual implementation.

Step 2: File I/O in golang

The first example we have, is creating a file. Here is how to create ‘test.txt’ and avoid a memory leak. A memory leak is when something is reference by the program, but the code flow of the program no longer has context for that reference.

// Get a read/write file reference by creating it if it doesn't exist, or appending to it if it does exist.
file, err := os.OpenFile("test.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666) // This provides a reference to the file 'f', or an error if it cannot. 

// check for the error. 
if err != nil { // a 'nil' error means no error occurred. 
  fmt.Println(err) // print the error
  return // stop processing the context.
}

// ##IMPORTANT## Always close the file reference to prevent a memory leak!
defer func() { // defer is an instruction that runs when the 'context' closes.
	_ = file.Close() // close the reference!, the underscore is a way to ignore the value returned by Close()
}()

Reading from an open file reference

// Reading each line in a file into an array
var lines []string // declare the slice to hold the data
scanner := bufio.NewScanner(file) // use the buffer input output package to start a file scanner
for scanner.Scan() { // loop through the file scanner
  lines = append(lines, scanner.Text()) // add the scanned line to the lines slice.
}

Writing to an open file reference

text := "Coming to a file near you" // string to write to file
_, err = fmt.Fprintln(file, text) // write the data to the file
if err != nil { // error handling if file I/O fails.
   fmt.Println(err)
}

\\ or alternatively
linesWritten, err := f.WriteString(text)

Step 3: Date storage format

There are many ways data can be stored within a file. If you’ve messed with files in the past – you’ll know a bit about text .txt files. Maybe you’ve seen markdown .md files. There are many different types of files – and these all serve different purposes. markdown, text, and even .csv files are all Human Readable Formats. If you open any of these in a text editor, you’ll be able to make sense of them quickly. However – they aren’t exactly the best for storing data from a program. For this particular example, we are going to simply use a txt file to place human readable data into the file.

Step 4: Chatbot Enhanced Memory

1. launch program
2. check if history exists (create it if it doesn’t exist)
   1. load the history into the history slice
3. send a friendly greeting and store it to history
4. send a simple instruction and store it to history
5. Have the user put something in and store it to history
6. echo what the user put in and store it to history
7. Check if the user typed ‘history’
   1. Store the request to history
   2. don’t repeat the history in the history file, just count the number of lines.
   3. Loop through the history slice
   4. print the key and the value
8. repeat 4-7 until the user says ‘bye’

Step 5: Putting it all together

Now that we’ve covered the details, we can run the program. We’ll find that it isn’t much different from running the program during lesson 2. However – when we say ‘bye’ and start the program again, that’s where the changes are noticeable.

1. run the program
2. type hello
3. type history
4. type bye
5. run the program again
6. type history

Here is the final example used above. It is also available in github in chatbot, lesson 3

func main() {
	f, err := os.OpenFile("history.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)

	// check for the error.
	if err != nil { // a 'nil' error means no error occurred.
		fmt.Println(err) // print the error
		return           // stop processing the context.
	}

	defer func() {
		_ = f.Close()
	}()

	// string for storing input
	var input string

	// Slice for storing the history of the world.
	var history []string
	scanner := bufio.NewScanner(f) // use the buffer input output package to start a file scanner
	for scanner.Scan() {           // loop through the file scanner
		history = append(history, scanner.Text()) // add the scanned line to the lines slice.
	}

	fmt.Println("Hello, World!")
	fmt.Println("I am Echo! Please tell me something to say by typing it in, and pressing enter!")
	_, _ = f.WriteString("Hello, World! \n")
	_, _ = f.WriteString("I am Echo! Please tell me something to say by typing it in, and pressing enter! \n")
	// Stop when we see "bye"
	for input != "bye" {

		fmt.Print("You: ")
		scanner := bufio.NewScanner(os.Stdin)
		for scanner.Scan() {
			input = scanner.Text()
			break
		}
		_, _ = f.WriteString("You: " + input + "\n")
		// Add the new input to the history
		history = append(history, input)
		fmt.Print("Echo: " + input + "\n")
		_, _ = f.WriteString("Echo: " + input + "\n")
		// Check if the input was the string 'history'
		if input == "history" {
			// We won't add the history to the history, instead we can add how many lines of history exist.
			_, _ = f.WriteString("<Truncated> " + strconv.Itoa(len(history)) + " Lines of history repetition.")
			// iterate (loop) through the history
			for k, v := range history {
				// print out the Key and the Value
				fmt.Println(k, v)
			}
		}
	}
}

I don’t know about you – but that code is starting to look like my pantry after my kids try finding the peanut butter. In our next lesson – we will start to look at organizing code a little bit better.

Many people can recount their first memory. We don’t really know how we do it, or why it is a first memory. It’s just there. We can also remember things more recently – or – remember information that is imaginary. For this lesson, we will be covering memory in terms of programming. Because a chatbot without a history, is like playing Memory and just choosing random cards each turn. Lesson examples can be found on Github

Step 1: Understanding the Lesson

Before we dig into code – let’s see if we can understand a little more about storing things in program memory. For most languages a developer can write, there are a few different ways to store data. In previous lessons, we’ve covered what a variable is – including a few types of variables. In this lesson, we are diving more deeply into the array type of variable.

One way to understand an array, is to think of series of mailboxes. They all have a number on them so the delivery person can put the mail into the correct slot. When a large neighborhood is built, there are only so many houses – and – there are only so many mailboxes. Depending on how this neighborhood is setup, any new houses will either be an easy addition (or) for a grouped mailbox setup – a new group would eventually need to be created. Arrays are like the grouped mailbox setup.

But what is an array, really? An array is a declared list of keys and values. From the mailbox example, the mailbox number is the key, and the things inside the mailbox are the value. At a more physical layer – arrays are blocks of ram reserved for storing sets of specific types of variables. In different programming languages, there are different mechanics for using arrays. In c++, one needs to program what happens if you try to add more keys to an array then the original requested amount. In golang – the language itself handles this for you using slices – so you don’t have to program the tedious memory management. Most modern languages handle the array management operations in some similar fashion.

Step 2: Using an Array in Golang

Now that we’ve briefly covered the definition of an array, let’s look at a few array operations. The Golang Tour provides an example of arrays here: A Tour of Go (golang.org).

Example from the golang tour with notes:

func main() {
  // creating a new array
  var a [2]string

  // adding to an array using a key
  a[0] = "Hello"
  a[1] = "World"

  // looking up something in an array
  fmt.Println(a[0], a[1]) // Prints Hello World

  // Creating an array with values
  primes := [6]int{2, 3, 5, 7, 11, 13}
  fmt.Println(primes) // Print [2 3 5 7 11 13]

  // removing the item with key 3 from the array
  var slice []int                                // create a 
  slice - notice how it doesn't have a length specified?
  slice = append(slice, 1, 2, 3, 4, 5, 6) // add values 
  to the array
  removalKey := 3                                       // set a key for removal

  // remove an item by adding the first portion (before the key, and the second half after the key together) 
  slice = append(slice[:removalKey], 
  slice[removalKey+1:]...)
  fmt.Println(slice) // Print [1 2 3 5 6]        // notice how the '4' is gone?

  // iterate (loop) through the slice and print the key and the value
  for key, value := range slice {
    fmt.Println(k, v)
  } 
}

Step 3: Chatbot wants a slice

With only a few lines of change, we are going to have chatbot keep a short term memory of the conversation. Here is a breakdown of the logic we’re going to use for chatbot:

1. launch program
2. send a friendly greeting
3. send a simple instruction
4. Have the user put something in
5. echo what the user put in
6. Check if the user typed ‘history’
   1. Loop through the history slice
   2. print the key and the value
7. repeat 4-6 until the user says ‘bye’

To start, we will initialize a new slice.

// Slice for storing the history of the world.
var history []string

Next, whenever we get new input, we are going to store it in the history

// Add the new input to the history
history = append(history, input)

Finally, chatbot is going to check for a ‘prompt’ that it will use to print the history.

// Check if the input was the string 'history'
if input == "history" {
	// iterate (loop) through the history
	for k, v := range history {
		// print out the Key and the Value
		fmt.Println(k, v)
	}
}

Step 4: Putting it all together

The completed above code can be found in this github repository. There is a main.go file (the code) and a README.md file (the directions).

To get the following output, run the program, and enter the following lines

1. type test
2. type cool
3. type history

Well, we’ve now got a chatbot that is still a copycat. It can now recount all the things it has copied. This ‘history’ only lasts while the program is running. In future lessons, we can explore more permanent memory, command logic, and making the code organized.