Creatively using text-files with an SD Card on a 3.5 inch TFT LCD Display and Arduino (Part 3 of 6)

The SD Card should be formatted with a FAT16 or FAT32 filesystem, with a single partition only. This is a mandatory step to perform, without necessarily getting into the technical details of formatting. Briefly, formatting refers to the process of configuring a storage medium so that it can be read-from/written-to by controllers (such as your PC or the Arduino). This involves "laying-out" a filesystem on the medium and partitioning it. You can read more about the history of formatting here.

There are a vast number of tutorials online that already explain how to do this, but a brief summary of the steps are shown below for different operating systems. Irrespective of your operating system, you will need an SD Card slot on your Laptop/PC/Mac, or an external SD Card reader if one is not available.

The library used to communicate with the SD Card is the SD Library that is included with the Arduino IDE by default. On PlatformIO, this can be included by adding arduino-libraries/SD to the lib_deps section. The following snippet shows the basic usage of the SD Library -

Note that the SPI library is included before the SD Library. This library is also included with the Arduino IDE by default.

First, the SD object is initialized in the setup function using thebegin method. It accepts a single argument - the chip-select pin of the SD Card (in case of the ILI9486 LCD Shield, this is pin 10). The method returns true if the initialization was successful and false otherwise. This should be checked before using the SD object further.

Next, the exists method is used on the SD object to check if a file exists. It accepts a single argument - the name of the file (as a string) to find. It returns true if the file exists and false otherwise.

After this, the open method is used on the SD object to open a file for reading. It accepts two arguments, which are the name of the file (as a string) and the mode to open it in. The latter can be either FILE_READ (allows reading only; does not create the file if it does not exist) or FILE_WRITE (allows reading and writing; creates the file if it does not exist). The method returns a file handle of type File, which refers to the opened file.

In an if-statement, the handle evaluates to true if the file was successfully opened and false otherwise. This should be checked before performing further operations on it. Note that the FILE_WRITE mode opens the file in append mode if it already exists, i.e. existing data is not cleared when the file is written to.

Files on the SD Card can be treated as a sequential array of bytes. Each file has a size, which corresponds to the number of bytes in the array. The file handle keeps track of this as well as the location of the current character being processed.

This way, the file handle can be used in a manner similar to the Serial object. Methods such as write, print and println can be used to write to the file. Whenever a character is written to the file, it is placed at the current location and the location is incremented (this happens automatically). To manually get and set the current location, the position and seek methods can be used.

To read from the file, the read and peek methods can be used. Theread method returns the character at the current location and automatically increments the location, while the peek method returns the character without incrementing the location. To check if the file has any characters left to read, the available method can be used.

After using the file, the close method can be used on the file handle to close it, which also saves all changes made to it. Detailed documentation for the SD Library is available at https://www.arduino.cc/reference/en/libraries/sd/.

In the coming sections, we will see more about the SD Library and its usage with the display and touchscreen to build useful programs.

Before moving onto writing programs, it is necessary to know the constraints imposed by the library (or by the FAT filesystem) on filenames -

1. Filenames can be no longer than 12 characters, including the extension. Any files with longer names will have their name truncated by the library when being referred to. If multiple files exist with long names and matching prefixes, the library will follow undefined behavior while truncating their names. Consequently, it is recommended to keep all filenames below 12 characters. In case of nested files (within directories), the name of each individual directory/file in the path should be less than 12 characters long (their combined length may exceed 12 characters).
2. Filenames are not case-sensitive, including the extension. The library changes all names to upper-case while referring to files. If a mixed-case/lower-case name is provided to any of the methods/functions, it gets converted to upper-case before getting used. If a file exists on the SD Card with a mixed-case/lower-case name, it is treated as being in upper-case by the library.
3. The root of the filesystem on the SD Card is referred to with /. Because of this, a file named 1.TXT that lies in the SD Card can be referred to as either 1.TXT or /1.TXT, and another file named2.TXT in a folder named nested can be referred to as either/nested/2.TXT or nested/2.TXT.

Let us now see an example of how configuration data can be stored on the SD Card separately from the program. Configuration data refers to any data which controls how the program runs, similar to thesettings on an Android phone. This data is read by the program and used to determine how it should execute. In many cases, it is also possible for the program to update the configuration data for future executions (by taking input from the user).

Consider a program that draws three squares on the display, using colors provided in a configuration file. The colors of the squares can be changed by changing the contents of the configuration file, without modifying the program.

Assume that the file is named CONFIG.TXT. It must follow the format shown below -

The placeholders c1, c2 and c3 must be replaced with actual colors, such as -

The colors shown above are hex representations of the 16-bit color codes for red, green and blue. This method of encoding colors was shown in the first part of this series. Ideally, the program should also be robust enough to handle the following input -

The second example includes extra spaces, irregular cases, and even C-style comments at the end of each line. Let us see a program that can use this information in this file to draw the squares.

Start by creating a new project and adding the constants.h file to it. Add the following code in it -

The procedure to create the file, as well as its explanation was given in the second part of this series. In brief, it contains constants that are used throughout the program, and are kept separate from the main file for the sake of readability. The constants are divided into three categories - those used to operate the display, the touchscreen, and application-specific values. The above snippet does not include any constants in the third category, as our program does not need them.

Moving onto the main file, add the following code to it -

The above code uses the SD Library to open and read the configuration data from an SD Card plugged into the TFT LCD Shield. It then uses this information to draw the squares on the display and color them. Let us look at it more closely.

The complete code can be found here on Github

Parsing the configuration file

---

The parse_config function is used to parse the config file and assign values to c1, c2 and c3. It starts by declaring the two arrays named key and value, which are used to store the key and value present on each line respectively. The key is the name of the property which is present before the =, while the value is its assigned value which is present after the =. Next, it contains a loop to iterate over the contents of the file, which can broadly be divided into 7 parts.

The first part is an inner loop that skips all characters till the first double-quote ("), which is the beginning of the key. If the end of the file is reached during this (indicated by read returning -1 instead of a valid character), there are no more properties left to read and the function returns.

The second part is another inner loop that reads the name of the key into the key array one character at a time. It keeps reading till a double-quote (") is encountered. If the end of the file is reached before a ", the invalid_config function is called to indicate that the file does not follow the proper format. The third part is yet another loop that skips all characters till the equals (=) character. As before, the file is considered to be in improper format if the end of the file is reached before an = character.

The fourth and fifth parts are for reading the value associated with the key and are analogous to the first and second loops. The fourth loop skips characters after the equals character till a double quote is encountered. The fifth loop reads the value one character at a time into the value array till the next double quote is encountered. If the file ends while either of these loops are executing, it is considered to be in an improper format.

After this, the contents of the key and value arrays are printed on the Serial Monitor for debugging. Finally, the name of the key (in the key array) is compared against known values to get the variable it corresponds to. After this, the hex number inside value is parsed into the selected variable. If the value is not a valid hex value, then the config is considered invalid.

After removing the SD card and uploading the program to the Arduino, you should see the following result -

After creating the CONFIG.TXT file on the SD Card and inserting it into the shield, you should see the following result (reset the Arduino after inserting the SD Card) -

If the second example were used instead, the following is shown -

After finishing this example, you could try to extend the program to make the coordinates and dimensions of the squares configurable as well, and even make the number of squares themselves configurable. This section was only intended to provide a starting point and introduce configuration files and their merits.

Let us now see an example of how data can be logged on the SD Card. Data logging is the process by which a set of events are recorded for the purpose of future analysis or diagnostics. For our demonstration, we will log the data given by the touchscreen.

This example polls the touchscreen till a touch is present. Once a touch is available, it saves the coordinates and timestamp of the touch to the SD Card every second. The data is stored in CSV format. The timestamp, x-coordinate and y-coordinate are present in different columns. Each entry is present on a new row.

As before, start by creating a new project and adding theconstants.h file to it. The contents of this file and its explanation was given earlier in this blog, while talking about configuration data.

Then, add the following code into the main file -

The above code waits for a touch to become available, and starts sampling the coordinates every second until the touch is removed. Let us look at it in greater detail.

The complete code can be found here on Github.

The loop function continuously polls the touchscreen for the presence of a touch. When one is detected, it reads its location and converts it to pixel-coordinates. After this, the LOGS.TXT file is opened in write mode and the color of the square is changed to red, indicating that it is not safe to power-off the device.

The timestamp (calculated using the millis function), x-coordinate and y-coordinate are appended to a new line on the file before closing it and changing the color of the square back to green. Finally, a delay of 1 second is added before polling the touchscreen again. This delay is added to prevent the Arduino from overloading the SD Card with values. It also means that tapping on the screen fast will cause some touches to get skipped.

When running the program, you will discover that the write process is incredibly fast and it is barely possible to observe the square changing colors.

The snippet shown below is an example of what the log file can contain after touching the screen at 4 points -

This example introduces how data can be logged for later analysis, and aims to provide a starting point for readers to build on.