Debugging on Much Ado About IT

Rules for working with dates and times

Julian Knight — Sat, 07 Mar 2020 15:35:49 +0000

Dates and times are incredibly complex. They can be different in different countries, languages. They are often inconsistent and have weird edge-cases. These are some basic rules I apply when working with dates and times.

when handling dates and timestamps, here are the foundation rules that I work to:

Always use ISO date format where possible (YYYY-MM-DD)

So that there can be no ambiguity and so that date strings will naturally sort
Always work in UTC (Aka Zulu time or GMT) except when displaying to the user

To avoid weird errors, especially with timezone and daylight savings transitions. It also makes date/time calculations a LOT easier.
When needing a string format for timestamps, always use ISO format

e.g. 2020-03-07T15:27:46.123Z

These are always unambiguous and easily machine parsed.
When specifying decimal seconds in a timestamp, avoid more than 3 decimal places

The parsing tool used in node-red-contrib-moment to parse input dates in different formats cannot cope with more than 3dp and may return strange dates, a problem that I reported in May 2019 but hasn’t been resolved.

It should also be noted that I’m not sure that MomentJS can cope with more than 9dp.

Also worth noting that JavaScript’s Date native object only supports up to 3dp anyway and will turn anything beyond that into zero’s.

References

parseFormat function issue for >3dp seconds resolution

The worst thing about this issue is that it can result in an incorrect date being returned but does not give a warning.

Cleaning up npm package install problems

Julian Knight — Wed, 05 Jun 2019 07:05:39 +0000

Node-RED uses npm to manage package installations such as its nodes. Sometimes, you can get into a situation where you start getting lots of npm installation or removal issues and it feels like you are going round in circles trying to fix things. If so, time to call it a day and to get drastic! Here, I try to explain an easy way to fix things quickly and completely without side effects.

Node-RED is built over Node.js which uses npm as its library (AKA package) manager. It is great to have a standard mechanism to manage libraries but the complexities of managing JavaScript based packages sometimes trips things up.

Once you’ve tried for a bit to understand what the issues are, it may be time to call time and get drastic.

The following commmands will clean out the packages installed for your live instance of Node-RED - assuming you have it installed via the “standard” default method. If not, you will need to adjust the folder you operate in.

cd ~/.node-red
rm package-lock.json
rm -R node_modules
npm install

That will cleanly reinstall all of the packages marked in your package.json file. So make sure that is correct first. Every time you install a package (as long as you have reasonably up-to-date versions of Node.js and npm) whether from the NR admin ui or command line, it gets added to package.json.

I now always remove the lock file as well if I’m having issues as that can be the cause of some issues. The lock file isn’t generally needed in NR installations.

Note, as always, I’ve assumed you have a “standard” install of Node-RED which puts your userDir as ~/.node-red (and the equivalent on Windows). If you use an installation scheme like mine, it will be in a different place as noted by the command used to start NR.

Analysing the Performance of Node-RED

Julian Knight — Fri, 19 Apr 2019 13:23:06 +0000

Node-RED is a powerful platform for prototyping and rapid development but it is complex. It has a lot of moving parts and it can be hard to know what might be causing performance bottlenecks. This article tries to give some hints and tips about analysing those bottlenecks and how to resolve them.

Sorry, this is draft and far from complete. Please leave comments if you have something useful to add. Thought I would publish anyway in case anyone finds it useful. I will try to come back to it in the future. If you leave a comment, it will remind me. :-)

Platform

Before starting more detailed analysis, the first thing to do is to look at the platform being used to run Node-RED.

Often, something like a Raspberry Pi or other single-board computer (SBC) is being used. They have very limited resources and often slow storage (e.g. SD-Cards). So it is very easy to overload them and kill performance.

To analyse this, you need to use a tool such as top. Better still, get something a bit more comprehensive such as glances. Either way, these will tell you how much CPU and memory (RAM) is being used and by what. Start by looking at the three ’load’ values, these should all be fairly low, if they are going over about 2-3 regularly then you have an issue.

Next look at the ‘swap’. If this is being used regularly, you will have a performance bottleneck because memory (RAM) is being overused. To fix this, you have to unload some services. If using the glances tool, it will show a warning if the memory is swapping.

The easiest way to remove load is likely to be turning off the ‘desktop’. This gives you a GUI that you directly interact with via keyboard and monitor. Removing this will remove lots of load on memory and CPU. Have a look at articles that explain how to run your device ‘headless’. You will need to be able to connect to your device over the network from another device.

After that, if there are still issues, you need to look at taking off services that you don’t actually need running.

Finally, if you still have issues, you will need to think about splitting services between multiple devices or using a more powerful device.

Back End (server)

Using Node.JS `--inspect` option

Front End Browser (Node-RED Admin UI)

For any browser performance issues, you firstly need to ensure that the problem is actually coming from something related to Node-RED. So you should turn off any browser extensions before continuing. An extension such as SimpleExtManager for Chromium browsers will assist with easily turning on/off extensions.

Browser Performance Profiling

Using your browser’s developer tools, you are able to record and analyse a performance analysis. The Chromium developer tools have three tabs to help with this. Performance, Memory and Application.

The Performance tab is perhaps the easiest to get going with. Click on that tab and then start profiling. Leave that for an amount of time that you estimate will let you see any ongoing performance issues. That might be just a few seconds or a few minutes. You will want to tick the “memory” option and probably don’t need the “Screenshots” option.

The thing to look at first is likely to be the memory utilisation. There is a graph 1/2 way down the display that shows you some important metrics including the “JS Heap”. See how big that is getting by hovering over the line. Also look at how fast it is growing. You should see that the heap reduces periodically as the JavaScript engine does a ‘garbage collection’.

You can also do multiple recordings and compare the heap over a longer period.

Front End (Dashboard, uibuilder, http-in/-out nodes, etc.)