spectro… adventures in Go (lang)
5 stars based on
My latest spare time project has been writing a general purpose command line heat map tool designed to visualise the distribution of streams of decimal numbers not necessarily integers representing something like latency, duration or size. It was inspired by this Sysdig tweet for a monitoring tool with built in command line visualisation, and follows on from my distribution Awk script which displays an actual histogram although it also has some real-time functionality.
You've never seen a command line tool do this. Visualizing real-time latency spectrograms: These things combined make it an interesting choice for writing command line tools.
You can be sure the tools will work on your system, as they are compiled by go get for your environment. You can be sure that the programs will not be affected by changes in your system packages, as the dependencies are contained in the executable it bakes. The end result was successful, although I did later find an existing alternative although this does not do everything I join the binary autotraderfrom below banner ####, namely replaying existing log files.
It provides some assistance with built in types, very basic refactors and syntax checking. Go has an interesting syntax, and a few quirky features some good, some hmmm. While I may not agree with all the decisions, it is clear that the language features have been given a lot of thought and display a lot of wisdom. For example, I particularly liked that since map iteration order is not guaranteed, it is actively randomised at runtime to make that clear.
I think the most frustrating part of the language was the constant conversion between various ints, floats etc. The definition of constants in Go—arbitrary precision values free of signedness and size annotations—ameliorates matters considerably, though. So a trade off has been made against clean code to simplify the compiler, ensure portability and mitigate overflows. But I would have loved not to have to cast an int returned by an external function to a 64 bit int where there is no risk of overflowsimply so my code could be less cluttered like the snippet below.
You can use the provided normal command to generate some test data based on a normal distribution:. Using dtrace to simulate some real time data for spectro to plot like this: But it really comes into its own if you have historic logs with a formatted time in the line. Here you can use the pacemaker command to indicate to spectro how to sample the data. The pacemaker command will add extra lines join the binary autotraderfrom below banner #### the streamed output as a signal to the spectro command.
Pacemaker will look for a time matching something like this: You can leave the time text in the output, so long as the number you wish to visualise is the last thing in the line. You can run something like cat sample. You can use pacemaker -nowait to analyse the file as quickly as possible otherwise it sleeps between timesand if the log file is still being written to you could just use tail -f sample.
I appreciated the line by line output of sysdigespecially as it allows the user to scroll back as far as their screen history allows. It also means you only have to modify one line of text at once, rather than re-plotting the whole screen. I decided early on I wanted to find a decent gradient of ANSI colours which covered the visible spectrum.
And the rainbow scheme is very attractive when testing with a normal distribution. Frustratingly though, it turned out that lots of colours is actually very difficult to read with a less organised distribution, so I limited it to blue through red.
I was keen to allow data to be streamed in real time, or replayed via an existing log file. I was join the binary autotraderfrom below banner #### keen that the charts would adapt to changes in input, join the binary autotraderfrom below banner #### that it is not necessary for the user to provide configuration flags.
If the range of inputs changes, the scale will be replotted an a message printed. Similarly, if the number of points in each bucket changes the magnitude scale will adapt. It is however possible for the user to start join the binary autotraderfrom below banner #### two values at appropriate levels to prevent constant changes.
I pulled the log replay functionality out into a separate command named pacemaker as I can see myself using it again in the future, and also to follow the Single Responsibility Principle and the join the binary autotraderfrom below banner #### of the Tenets of the UNIX Philosophy: Communication between pacemaker and spectro is via extra plain text lines to keep things readable, and both are designed to work as filters in a pipeline tenet nine.
I think I would recommend using Go for a similar task in the future, simply for the cross platform distribution mechanism, lack of dependency issues and freedom from packaging pain. You can find out more, including install instructions at https: Developer, engineer, trouble maker. Hosted on Github Pages using Jekyll and Flex.
Tue Nov 11