Logging multiline strings

Simple answer – don’t log multiline string. You will loose or benefits coming from tools like grep. The philosophy of these tools is, that every line is a self-contained piece of information. Spreading the information across multiple lines breaks the concept.

So what to do with ‘\n’ or ‘\r’ characters? Replace (escape) them with them literally with “\n” or “\r”. Apply the same to ‘\t’ (tab) and ‘”‘ (double quotes) as well. For example a message like

Response: {
   code: 1,
   message: "No errors found"
}

would by written to a log message like this:

[2015-01-03 16:26:23.136049] ERROR: Response: {\n    code: 1,\n    message: \"No errors found\"\n}

An example of a logging library that enables this behavior is https://github.com/Seldaek/monolog for PHP.

Now it is possible to grep such logs, but they don’t look pretty and are hard to read visually. Here is how to escape back previously escaped characters while tailing multiple logs.

tail -f /var/log/httpd/error_log /var/log/httpd/vhost15/error_log | grep -v " \[info\] " | sed 's/\\r\\n/\n/g; s/\\r/\n/g; s/\\n/\n/g; s/\\t/\t/g; s/\\"/"/g;'

We are using Linux command sed, which is basically a command line text editor. Traditional text editors are interactive, they come with a GUI, where text is displayed, user moves a cursor around and types down the text. Some editors allow to record macros. Macros are small programs and allow to repeat repetitive tasks. sed is a text editor with macros, but without GUI. Because it is a command line utility, it can be combined with other command line utilities, e.g. with tail

In this case, we use sed to recognize simple regular expressions and replace them with other characters. More concretely, we replace escaped Windows EOL (\r\n) with a simple UNIX EOL, escaped Mac EOL with UNIX EOL, escaped UNIX EOL with UNIX EOL itself too and escaped TAB with TAB itself.

Leave a Reply

Your email address will not be published. Required fields are marked *