Development on Much Ado About IT

Node-RED communications protocols

Julian Knight — Thu, 20 Aug 2020 17:53:20 +0000

Node-RED has an excellent choice of communications protocols available to it. This article looks at the main options and their strengths and weaknesses. It also gives suggestions as to when you may wish to use them.

Node-RED is renown for its MQTT support since that is where it started - as an IBM demonstrator for their experteese on IoT.

However, this is only one of many ways to send/receive data between Node-RED and other systems or services. Each of tese protocols has its own advantages and disadvantages.

The protocols

In rough order of descending order of ease of use (very rough!):

MQTT
Websocket
HTTP
TCP
UDP
UNIX Pipes

MQTT

This is likely to be the easiest to use even though it requires an extra service to be running, a “Broker”.

There is even a custom node available that lets you run a broker from within Node-RED itself. However, I don’t recommend doing that because it adds significant overheads that are easily avoided. The most commonly used broker is “Mosquito” which has low resource overheads, is fast, secure and capable. It is easily installed and configured.

While MQTT data payloads are just strings, Node-RED lets you send and receive more complex data structures (JSON).

The things that put MQTT at the top of the list are:

The ability to easily handle more complex data
The availability of MQTT libraries and clients for many platforms

Websocket

HTTP

TCP

UDP

UNIX Pipes

How to reliably detect the object type of a JavaScript variable?

Julian Knight — Sun, 07 Jun 2020 14:30:54 +0000

The nature of JavaScript means that everything is treated as an object (sort of). It can be remarkably hard to reliably detect what type of object something is and there isn't a single function in JavaScript that can be used in all cases. So how do we do it? Here are some options

The “official” way is to use Object.prototype.toString.call(objectToTest). This will work no matter what the object is, even null or undefined. However, it is relatively quite slow.

A quicker way is probably to use objectToTest.__proto__.constructor.name. However, note that this will fail (as do many other methods) if the object is null or undefined.

Simple function

This simple function provides an easy to use way to return the object type and a list of simple type ENUM’s to compare against.

Note that using Object.prototype.toString.call(prop) is a fairly slow operation and therefore this should be avoided when maximum performance is required.

var types = {
 'get': function(prop) {
 return Object.prototype.toString.call(prop)
 },
 'null': '[object Null]',
 'object': '[object Object]',
 'array': '[object Array]',
 'string': '[object String]',
 'boolean': '[object Boolean]',
 'number': '[object Number]',
 'date': '[object Date]',
}

Used as:

if(types.get(prop) == types.number) {
 // ...
}

It can, of course, be extended with your own bespoke object types.

This is taken from an answer to the StackOverflow Question: The most accurate way to check JS object’s type?.

FAQ 4: How do I see the Node-RED log?

Julian Knight — Tue, 17 Mar 2020 16:28:07 +0000

Just as there are several possible ways to run Node-RED, there are several ways to access the log.

In line with most server type applications, Node-RED outputs all sorts of operational and error information to a log. In addition to Node-RED’s own logging, each node that you use may also add to the log.

You can also output your own information to the log from your flows, those methods are shown at the end of this article.

There are three categories of ways to actually run Node-RED and each will require you to access the log in a slightly different way.

Running Node-RED as a “Service”

If you install Node-RED using the script supplied for installation on a Raspberry Pi (or any other Debian-like host), Node-RED will be run as a system service. That is to say that it will be started up when the system boots. You can also do this on other platforms including Windows.

Linux

In these cases, typically system log output goes to a standard service such as syslog. Typically, the syslog output is found in a file at /var/log/syslog and you can use the tail command from a terminal command prompt to view it.

On most modern versions of Linux though, the syslog is being overtaken by logging built into the systemd service. In these cases, it is better to use the journalctl command to show the log output. This has the advantage that you only need to know the service name (node-red on a standard Pi install) and you don’t need to worry about where the actual data is stored. You can also use the more advanced features of that command to output logs in different formats.

The following command will show you the last 50 lines of the log, jumps to the last output and “follows” any further output.

journalctl -u node-red -e -f -n 50

Windows

Windows does not use log output in the same way that Linux does as it has the “Event Logs”. Typically, log output is redirected to a file and you will need to look at the startup command for Node-RED to see where the logs are being written to.

Running Node-RED manually

On all platforms, if you run Node-RED manually from the command line, the log output is shown in the same terminal. Using the keyboard command ctrl-c will both stop Node-RED and close the log.

Using a “runner” to run Node-RED

Sometimes, especially during development, it is helpful to use a “runner” application such as PM2 or nodemon to start Node-RED. These will recover from failures and can be made to “watch” for changes to files and restart automatically.

Each of these tools may have their own ways of viewing log output. For example, when using the popular pm2 runnner, the command is pm2 logs.

Changing log output levels

By default, Node-RED will show the following types of output from both Node-RED and any installed nodes:

fatal - only those errors which make the application unusable should be recorded
error - record errors which are deemed fatal for a particular request + fatal errors
warn - record problems which are non fatal + errors + fatal errors
info - record information about the general running of the application + warn + error + fatal errors

You can change the logging levels to also show:

debug - record information which is more verbose than info + info + warn + error + fatal errors
trace - record very detailed logging + debug + info + warn + error + fatal errors

Or you can remove levels or turn off logging alltogether (using the off setting).

In addition, there are 2 other settings that impact logs:

metrics - Whether or not to include metric events in the log output
audit - Whether or not to include audit events in the log output

All of these settings are found (complete with instructions) in the settings.js file typically found in the userDir folder (normally ~/.node-red/settings.js).

Outputting your own information to the log

When writing flows in Node-RED, you will normally use the debug node that, by default, outputs to the debug sidebar in the Node-RED Editor (admin UI). However, you can change the settings in the debug node to also (or instead) output to the log.

In addition, you can also output to the log from a function node using the code:

node.warn('some useful text')

Output:

Mar 17 17:12:48 pi node-red[18577]: 17 Mar 17:12:48 - [warn] [function:My useful fn node] some useful text

If you give your function node a reasonably name, you will see that in the output as shown. If the node doesn’t have a name, the id will be shown, you can use that in the Editor’s search feature to find the node.

FAQ 3: What different ways are there to install Node-RED and which is best?

Julian Knight — Tue, 10 Mar 2020 09:26:00 +0000

There are several possible ways to install Node-RED, this post provides an overview and some thoughts about when you might want to use which approach.

There are three basic approaches to installing Node-RED. Which of these is “best” for you will depend on your knowledge, willingness to learn, desire or need to gain maximum flexibility and so on. Only you can work that out. It should also be noted that you should be able to mix and max these approaches if you really want to (though I can’t say that I would personally recommend that).

Global Install

This is the standard and therefore arguably by far the easiest installation.
It is good for people who are not overly familiar with their OS’s filing system or how node.js and npm work and who don’t want to be bothered to find out (that is not a criticism by the way).
It allows for multiple instances to be run if needed but all instances must use the same version of Node-RED.
It doesn’t “pollute” your home folder with the node-red and many other required node.js modules.
It is standard and most tutorials and explanations assume that this is the way it is installed.
It makes the node-red command global so you can run it from anywhere.

Advantages

Best for beginners unless you are prepared to learn about Node.js.

Disadvantages

Less flexible and possibly a little less secure. Not the best for “production” installations. Can confuse beginners into installing other Nodes and packages into global when they shouldn’t. Hard to work out where the global files are.

Local Install

This is a more “node.js way” of installing - global installations being good for exposing global commands but may sometimes be less secure.
It is a more self-contained installation. Everything can be in 1 place under a single folder structure. This makes it easier to back-up everything together. But it doesn’t force you to work this way, you can keep your userDir folder separate if you like.
It allows you to run different instances of Node-RED using different versions. Good for testing major Node-RED upgrades and for testing custom nodes or flows against different versions.
You are in control. You know where everything is because you chose to put it there.
You control access to all files and folders and so this can be a more flexible installation. You could also have different instances of node-red on the same device with different folder/file access levels. Possible development and test instances on the same device.
A good way to create a more portable installation of Node-RED since the whole folder structure can be easily copied/archived, etc.
It is my personal belief that this is also the best approach for people who want to learn how Node.js and/or Node-RED works. But then again, maybe that’s just my warped mind!

Advantages

Best for flexibility and control. Good for control of security. Best for running different versions of Node-RED in parallel on one device. Good for ease of backup/recovery and for creating distributable copies of a Node-RED application. Best for embedding Node-RED into an existing or new Node.js app.

Disadvantages

Requires some knowledge of how Node.js and npm work.

Containerised Install

Uses either Docker or snap to containerise the installation.
Isolates Node-RED from the host OS.
Allow running multiple, independent instances of Node-RED, possible using different versions.

Advantages

Best for enterprise use where containerisation is already well supported.

Disadvantages

Adds significant hidden complexity and requires good knowledge of the container technology to isolate and fix issues. Also likely to have higher resource requirements than running separate instances in other ways. Experience from the Node-RED forum suggests that a decent knowledge of Docker/snap is recommended.

There is some additional information that I wrote in a recent blog post:

https://it.knightnet.org.uk/kb/nr-qa/faq2-multiple-instances/

Rules for working with dates and times

Julian Knight — Sat, 07 Mar 2020 15:35:49 +0000

Dates and times are incredibly complex. They can be different in different countries, languages. They are often inconsistent and have weird edge-cases. These are some basic rules I apply when working with dates and times.

when handling dates and timestamps, here are the foundation rules that I work to:

Always use ISO date format where possible (YYYY-MM-DD)

So that there can be no ambiguity and so that date strings will naturally sort
Always work in UTC (Aka Zulu time or GMT) except when displaying to the user

To avoid weird errors, especially with timezone and daylight savings transitions. It also makes date/time calculations a LOT easier.
When needing a string format for timestamps, always use ISO format

e.g. 2020-03-07T15:27:46.123Z

These are always unambiguous and easily machine parsed.
When specifying decimal seconds in a timestamp, avoid more than 3 decimal places

The parsing tool used in node-red-contrib-moment to parse input dates in different formats cannot cope with more than 3dp and may return strange dates, a problem that I reported in May 2019 but hasn’t been resolved.

It should also be noted that I’m not sure that MomentJS can cope with more than 9dp.

Also worth noting that JavaScript’s Date native object only supports up to 3dp anyway and will turn anything beyond that into zero’s.

References

parseFormat function issue for >3dp seconds resolution

The worst thing about this issue is that it can result in an incorrect date being returned but does not give a warning.

FAQ 2: How can I run multiple instances of Node-RED on the same server device?

Julian Knight — Fri, 06 Mar 2020 01:21:33 +0000

There are all sorts of reasons to want to run more than a single instance of Node-RED and several ways to do it.

Use the command line settings to point to a different userDir folder.

node-red -u /data/userDirectory

That lets you use different flows and configuration settings but only allows a single version of Node-RED.

You still install Node-RED in the “standard” way, globally. While this is generally simpler for complete beginners, it can lead to complexities later on. You should understand the strengths and weaknesses of npm global installs.
Install the Node-RED npm package locally rather than globally.

This gives you not only the features from 1 but also allows you to run different versions of Node-RED in parallel. The alternate-node-red-installer shows you how to do that.

It also removes the need to install Node-RED globally which can lead to unexpected security issues.

This is also the “Node.JS way” and likely to be a more welcome approach for organisations and teams that run other Node.JS services and systems.

While slightly more complex to think about initially, especially for Node.JS beginners, it helps you understand how Node.js works, is easier to work out where everything is kept and is likely to be easier to maintain in the longer term.

I use npm’s script capability to make things really easy to start manually. Here are some examples. Note that I always make the userDir folder a sub-folder of the master project folder (where Node-RED itself is installed). Keeps things neat and makes it trivial to backup the whole thing.
```
{
 ...
 "scripts": {
 "start": "node node_modules/node-red/red.js --userDir ./data",
 "start2": "set DEBUG=express:* & nodemon node_modules/node-red/red.js --userDir ./data",
 "inspect": "node --inspect node_modules/node-red/red.js --userDir ./data",
 "log": "sudo journalctl -u node-red -f -n 0 -o cat",
 "update": "npm install --unsafe-perm --production node-red",
 },
 ...
}
```
So you can start manually using npm start and start with debugging port active using npm run inspect
Run Node-RED in a container such as the Docker container.

Allows multiple versions of Node-RED along with different configurations and supporting software and services.

However, this comes at the cost of significantly greater complexity and resource overheads. Unlikely to be worth doing except in specific circumstances & needs a reasonable level of understanding of how Docker works.

You will realise that I much prefer method 2 for all of my own installations. That’s why I wrote the alternate installer which goes some way towards making installations easier and more consistent.

For running Node-RED on system startup, please see the Starting Node-RED on boot page in the docs.

FAQ 1: Should I use a function node or a flow of core nodes?

Julian Knight — Fri, 06 Mar 2020 01:16:35 +0000

There are usually many ways to achieve something in Node-RED. A common question is whether or when to use a function node or a collection of core nodes to achieve an outcome.

Whether I use one approach or the other depends on the complexity of the input vs the complexity of the output. It really doesn’t matter except that it is often easier to read a single function node that several complex change nodes as an example.

So my general rules of thumb for whether to replace a collection of nodes with a function node are:

How difficult would this be to do in a pure flow? If it is just a couple of nodes, that’s fine, if it is 10 nodes then probably not fine.
If I am using JSONata, how long is it taking me to puzzle over the JSONata code? If it is a few minutes then fine, otherwise use a function node!
Did someone already write some JavaScript that I can purloin? If so then use it!

Of course, I know JavaScript reasonably well and if you don’t then your own rules might look a little different.

A Glossary of Node-RED terms

Julian Knight — Fri, 21 Jun 2019 17:40:30 +0000

For those not used to it, the terminology used in Node-RED can occasionally be confusing. Due in part to the overloading of some terms. So I've come up with this glossary to help keep myself from getting confused.

A draft set of terminology for Node-RED, the flow-based visual programming tool.

Glossary

Editor: The Node-RED admin UI. Where you edit and deploy flows.
Flows: The collection of node instances and wires defined by the Flows file loaded by Node-RED at run time.
Flow: A tab in the Node-RED admin UI used to break Flows into more manageable chunks.
flow: A set of node instances and wires with in a Flow tab.
Package: An npm package used, amongst other things, to install Node-RED Nodes.
Node: The source definition for nodes that can be used in Node-RED Flows. Defined normally by an npm Package (though actually defined by 3 files: package.json, an html file for the admin UI and a JavaScript file for the server process).
Node Instance: An instance of a Node added to a flow. A Node (definition) may have many instances in Flows.
node(s): same as Node Instance(s).
wire(s): The logical connectors between nodes. Conceptually, messages “travel” between nodes. In reality, they don’t, data is simply passed by reference (except for the edge cases where a message copy is forced).
message(s): Often referred to by the short name msg which is the variable name generally used to reference the message. This is the data object used to communicate between nodes. Typically, it has the minimal definition: {{_msgid: uid, topic: undefined|string, payload: Object|Array|string|number}} but other than requiring _msgid (which is added by Node-RED), it may contain any valid, serialisable JavaScript.
userDir: The server folder that contains the Flows file, settings.js and other Node-RED configuration information. Typically residing at ~/.node-red.
Project: a Flows file and other configuration files that allow rapid switching between Flows. Note that a project shares the same set of Packages (and therefore Nodes) and global Node-RED settings as all other projects. Only the Flows file, matching credentials, a README and a minimalist package.json are part of a project. Projects are designed to enable teams to work together and can use GIT to manage the data collaboration.

Creating a notification in the Admin UI

Julian Knight — Sun, 16 Jun 2019 19:22:16 +0000

Node-RED's admin UI has a built-in notification (toast) pop-up message feature. This article explains how to use it when creating custom nodes as it isn't currently well documented.

RED.notify('Your message here', options)

This can be called from the script in your admin ui code <nodeName>.html.

By default, the notification will disappear after a few seconds.

Options

type: {success|warning|error=} Adds style to the message box.
fixed: {boolean=false} Defaults to false, the notification will auto-remove after a few seconds.
modal: {boolean=false} If true, no other user interactions are allowed until the notification has cleared.
timeout: {ms=5000}: Specify the time in milliseconds a non-fixed notification will be displayed for.
id: {string=} Allows a notification to be updated.
width: {px=} Desired width in pixels. (Will be limited to the parent width)

Methods

.close(): Close the notification (useful if fixed option is true).
update
hideNotification
showNotification

Example

let myNotification = RED.notify('My message in a box', {
 modal: true,
 fixed: true,
 type: 'warning',
 buttons: [
 {
 'text': 'cancel',
 'click': function(e) {
 myNotification.close()
 }
 },
 {
 'text': 'okay',
 'class': 'primary',
 'click': function(e) {
 myNotification.close()
 },
 },
 ],
})

References

https://discourse.nodered.org/t/node-red-admin-ui-display-message-to-user/12290
https://github.com/node-red/node-red/blob/ccc3809daa3b03e7171cb6f4ccf4d6e0f1188bef/packages/node_modules/%40node-red/editor-client/src/js/ui/notifications.js#L16
https://github.com/node-red/node-red/wiki/API-Reference (old but has some things not documented elsewhere)

Cleaning up npm package install problems

Julian Knight — Wed, 05 Jun 2019 07:05:39 +0000

Node-RED uses npm to manage package installations such as its nodes. Sometimes, you can get into a situation where you start getting lots of npm installation or removal issues and it feels like you are going round in circles trying to fix things. If so, time to call it a day and to get drastic! Here, I try to explain an easy way to fix things quickly and completely without side effects.

Node-RED is built over Node.js which uses npm as its library (AKA package) manager. It is great to have a standard mechanism to manage libraries but the complexities of managing JavaScript based packages sometimes trips things up.

Once you’ve tried for a bit to understand what the issues are, it may be time to call time and get drastic.

The following commmands will clean out the packages installed for your live instance of Node-RED - assuming you have it installed via the “standard” default method. If not, you will need to adjust the folder you operate in.

cd ~/.node-red
rm package-lock.json
rm -R node_modules
npm install

That will cleanly reinstall all of the packages marked in your package.json file. So make sure that is correct first. Every time you install a package (as long as you have reasonably up-to-date versions of Node.js and npm) whether from the NR admin ui or command line, it gets added to package.json.

I now always remove the lock file as well if I’m having issues as that can be the cause of some issues. The lock file isn’t generally needed in NR installations.

Note, as always, I’ve assumed you have a “standard” install of Node-RED which puts your userDir as ~/.node-red (and the equivalent on Windows). If you use an installation scheme like mine, it will be in a different place as noted by the command used to start NR.

Analysing the Performance of Node-RED

Julian Knight — Fri, 19 Apr 2019 13:23:06 +0000

Node-RED is a powerful platform for prototyping and rapid development but it is complex. It has a lot of moving parts and it can be hard to know what might be causing performance bottlenecks. This article tries to give some hints and tips about analysing those bottlenecks and how to resolve them.

Sorry, this is draft and far from complete. Please leave comments if you have something useful to add. Thought I would publish anyway in case anyone finds it useful. I will try to come back to it in the future. If you leave a comment, it will remind me. :-)

Platform

Before starting more detailed analysis, the first thing to do is to look at the platform being used to run Node-RED.

Often, something like a Raspberry Pi or other single-board computer (SBC) is being used. They have very limited resources and often slow storage (e.g. SD-Cards). So it is very easy to overload them and kill performance.

To analyse this, you need to use a tool such as top. Better still, get something a bit more comprehensive such as glances. Either way, these will tell you how much CPU and memory (RAM) is being used and by what. Start by looking at the three ’load’ values, these should all be fairly low, if they are going over about 2-3 regularly then you have an issue.

Next look at the ‘swap’. If this is being used regularly, you will have a performance bottleneck because memory (RAM) is being overused. To fix this, you have to unload some services. If using the glances tool, it will show a warning if the memory is swapping.

The easiest way to remove load is likely to be turning off the ‘desktop’. This gives you a GUI that you directly interact with via keyboard and monitor. Removing this will remove lots of load on memory and CPU. Have a look at articles that explain how to run your device ‘headless’. You will need to be able to connect to your device over the network from another device.

After that, if there are still issues, you need to look at taking off services that you don’t actually need running.

Finally, if you still have issues, you will need to think about splitting services between multiple devices or using a more powerful device.

Back End (server)

Using Node.JS `--inspect` option

Front End Browser (Node-RED Admin UI)

For any browser performance issues, you firstly need to ensure that the problem is actually coming from something related to Node-RED. So you should turn off any browser extensions before continuing. An extension such as SimpleExtManager for Chromium browsers will assist with easily turning on/off extensions.

Browser Performance Profiling

Using your browser’s developer tools, you are able to record and analyse a performance analysis. The Chromium developer tools have three tabs to help with this. Performance, Memory and Application.

The Performance tab is perhaps the easiest to get going with. Click on that tab and then start profiling. Leave that for an amount of time that you estimate will let you see any ongoing performance issues. That might be just a few seconds or a few minutes. You will want to tick the “memory” option and probably don’t need the “Screenshots” option.

The thing to look at first is likely to be the memory utilisation. There is a graph 1/2 way down the display that shows you some important metrics including the “JS Heap”. See how big that is getting by hovering over the line. Also look at how fast it is growing. You should see that the heap reduces periodically as the JavaScript engine does a ‘garbage collection’.

You can also do multiple recordings and compare the heap over a longer period.

Front End (Dashboard, uibuilder, http-in/-out nodes, etc.)

A more comfortable way to edit the code for creating custom nodes

Julian Knight — Mon, 18 Mar 2019 17:38:40 +0000

Node-RED allows us to create custom nodes to extend the features. While this is a great feature, actually writing the code can be tricky. In part because the code for the admin configuration panel is contained in an html file containing 3 script entries that are Node-RED specific. This post shows you a relatively easy way to split that into 3 files and combine them with a build step using an npm cross-platform script.

When defining a new node in Node-RED, the code is contained in two files. An html file and a JavaScript file.

The JavaScript file contains the core code that is loaded into the server and does most of the hard work. As this is a standard js file, nothing special is needed to edit it, your favourite code editor or IDE will do the job nicely. My preference is VScode from Microsoft.

However, the html file isn’t very comfortable to edit because any HTML it actually contains is wrapped in some special <script> tags that are only recognised by Node-RED and they throw out your code editor making things really hard to read and process.

<script type="text/javascript">
... js that is run in the admin ui when the configuration panel is opened ...
</script>

<script type="text/x-red" data-template-name="uibuilder">
... html that is used in the admin node configuration panel ...
</script>

<script type="text/x-red" data-help-name="uibuilder">
... html that is used in the information sidebar ...
</script>

To get round this, I’ve started creating a separate ./node-src folder and splitting the content for this file into three source files: script.js, template.html and help.html.

Of course, you can call them whatever you like & will probably want to include the node-name if you are creating multiple nodes.

There is then a cross-platform executable that replaces the empty master with the contents of the three files.

Now all you have to do is to remember to run the build before pushing your changes to your repo and before publishing a new version to npm. That can also be automated as long as you are using npm to manage git. Look at the prepublishOnly script which will automatically allow you to run the build before publishing. You can do similar things with git by setting up the appropriate scripts.

Installation

To get this all working, you will need to firstly install a package called replace-in-file. Install this to the root of the folder holding you node.

npm install replace-in-file -D

Note that I install it as a devDependency because it is only needed for the node’s editors and not for end users.

Next, add the following to the package.json file, also in the root folder of your node.

 ...
 "scripts": {
 "build": "node ./bin/mergehtml",
 "prepublishOnly": "npm run build"
 },
 "bin": {
 "mergehtml": "./bin/mergehtml"
 }
 ...

This lets you run the build on any platform supported by npm, including Windows:

npm run build

Now, create folders ./bin and ./node-src. Into node-src, put the three files and your master template. Into bin put the mergehtml.js file shown below.

./bin/mergehtml.js

This is the code that does the magic. I overwrites the nodes html file (./nodes/node.html in this example) with the blank template then replaces the three <script> tags with the contents of the files from node-src.

#!/usr/bin/env node

/** Merge the various sections of nodes/uibuilder.html from the files in node-src/
 * Note that this is run from `npm run mergehtml` which puts the working directory
 * to the root, not `./bin`
*/

const replace = require('replace-in-file')
const fs = require('fs')
//const path = require('path')

// nodes\lib\uibuilder-help.html
const myhelp = fs.readFileSync('./node-src/help.html')
const mytemplate = fs.readFileSync('./node-src/template.html')
const myscript = fs.readFileSync('./node-src/script.js')

// Copy template - overwrite the nodes html file
fs.copyFileSync('./node-src/node.html', './nodes/node.html')

var options = {
 files: './nodes/uibuilder.html',
 from: [
 /(<script type="text\/javascript">)(.|\n)*?(<\/script>)/gmi,
 /(<script type="text\/x-red" data-template-name="uibuilder">)(.|\n)*?(<\/script>)/gmi,
 /(<script type="text\/x-red" data-help-name="uibuilder">)(.|\n)*?(<\/script>)/gmi,
 ],
 to: [
 `$1\n${myscript}\n$3`,
 `$1\n${mytemplate}\n$3`,
 `$1\n${myhelp}\n$3`,
 ],
}

replace(options)
 .then(changes => {
 console.log('MERGEHELP: data-help-name Modified files:', changes.join(', '));
 })
 .catch(error => {
 console.error('MERGEHELP: Error occurred:', error);
 })

console.log('MERGEHELP: Completed')

./node-src/node.html

Here is the content of the node.html file - you can use this for any project. You can put text between the tags if you like, even change the order of them if you prefer, the code should still work just fine. Obviously, any text you put between the open/close tags will be replaced.

<script type="text/javascript"></script>

<script type="text/x-red" data-template-name="uibuilder"></script>

<script type="text/x-red" data-help-name="uibuilder"></script>

How to take control of JavaScript console output

Julian Knight — Sat, 02 Feb 2019 00:52:46 +0000

All browsers have the console object that lets you output to the developer console log. But in production, you don't want this output, only in development. So here is the easiest way I've found to take control.

This is a simple example and really needs expanding to be more general purpose.

It uses a global debug variable to control output. If debug is false, no output will occur.

All functions of the console object are taken over by this function.

Code

/** Debugging function
 * @param {string} type One of log|error|warn|info|dir, etc
 * @param {...*} msg Msg(s) to send to console
 * WARNING: ...args is ES6, it doesn't work on IE11
 * @since 2019-02-01 Apply any number of args
 **/
//myDebug = function (type, ...args) {
myDebug = function () {
 if (!debug) return

 var type = arguments[0]

 //console[type](...args)
 console[type].apply(undefined, [].slice.call(arguments, 1))

} // --- End of debug function --- //

What JavaScript versions are there and how do they map to Node.JS versions?

Julian Knight — Fri, 04 Jan 2019 20:03:59 +0000

There are now far too many versions of JavaScript. Worse, there are several ways to refer to a JavaScript version. Then we have Node.JS which has its own versioning. This post tries to summarise it all.

This is just a summary for my own reference. See References below for more details.

Please let me know if there are any errors. I will update the table periodically.

ECMA-262 Version	ECMAScript Year Version	Publish Date	Node.JS Version
5	N/A	2009-12
6	ES2015	2015-06	6.5.0+
7	ES2016	2016-06	7.0.0+
8	ES2017	2017-06	8.10.0+
9	ES2018	2018-06	10.0.0+

Notes:

Versions of Node.JS prior to those shown are likely to support some features of newer JS versions. The versions shown are those that support virtually all of the features.
Some JavaScript features may not be implemented in Node.JS.
Node uses the V8 JavaScript engine. Also used by the chromium browser (which underpins Vivaldi, Google Chrome, Opera and many others. Soon to be used by Microsoft Edge as well).

References

Wikipedia ECMAScript Article
Kangax’s JavaScript Compatability Table
Node Green - Maps Node versions to JavaScript versions and shows what is supported in each version. Uses Kangax’s table.
Mozilla Developer Network (MDN) - Details each aspect of JavaScript as a reference, each page has a compatibility table showing browsers support. Uses Kangax’s table.

How to loop over JavaScript arrays and objects

Julian Knight — Fri, 04 Jan 2019 20:00:07 +0000

JavaScript can be a pain at times. Loops are a fundamental part of all computer languages but in JavaScript, there are some oddities. This post is a summary of the different loop features and when to use them. It is likely to be updated from time-to-time as the standards are still changing.

This is a personal, and incomplete reminder about doing efficient loops in JavaScript.

Of course, you should generally start with whatever is simplest. Generally you should not attempt to optimise too soon.

Arrays

Objects

The old fashioned way still works & is still generally the fastest.

var keys = obj.keys().sort()
for (var i = 0; i < keys.length; i++) {
 console.log(i, obj[keys[i]])
}

Somewhat faster version of the above (the length is not calculated on each loop and prefix iterator is slightly faster than postfix).

var keys = obj.keys().sort()
for (var i = 0, len = keys.length; i < len; ++i) {
 console.log(i, obj[keys[i]])
}

From JS v5, you can use a combination of Object.keys() and Array.prototype.forEach().

Object.keys(obj).forEach(function(key) {
 console.log(key, obj[key])
})

In addition, Object.values(obj) does what you’d expect.

for ... in (MDN) - includes non-own properties.

From JS v6 (ES2015), you can use for...of.

for (const key of Object.keys(obj)) {
 console.log(key, obj[key])
}

In JS v8 (ES2017), Object.entries() was added which avoids having to look up each value in the original object.

Object.entries(obj).forEach(
 ([key, value]) => console.log(key, value)
)

Object.keys() and Object.entries() iterate properties in the same order as a for...in loop. However, they ignore the prototype chain. Only the object’s own enumerable properties are iterated.

Development on Much Ado About IT

Node-RED communications protocols

The protocols

MQTT

Websocket

HTTP

TCP

UDP

UNIX Pipes

How to reliably detect the object type of a JavaScript variable?

Simple function

FAQ 4: How do I see the Node-RED log?

Running Node-RED as a “Service”

Linux

Windows

Running Node-RED manually

Using a “runner” to run Node-RED

Changing log output levels

Outputting your own information to the log

FAQ 3: What different ways are there to install Node-RED and which is best?

Global Install

Advantages

Disadvantages

Local Install

Advantages

Disadvantages

Containerised Install

Advantages

Disadvantages

Rules for working with dates and times

References

FAQ 2: How can I run multiple instances of Node-RED on the same server device?

FAQ 1: Should I use a function node or a flow of core nodes?

A Glossary of Node-RED terms

Glossary

Creating a notification in the Admin UI

Options

Methods

Example

References

Cleaning up npm package install problems

Analysing the Performance of Node-RED

Platform

Back End (server)

Using Node.JS --inspect option

Front End Browser (Node-RED Admin UI)

Browser Performance Profiling

Front End (Dashboard, uibuilder, http-in/-out nodes, etc.)

A more comfortable way to edit the code for creating custom nodes

Installation

./bin/mergehtml.js

./node-src/node.html

How to take control of JavaScript console output

Code

What JavaScript versions are there and how do they map to Node.JS versions?

References

How to loop over JavaScript arrays and objects

Arrays

Objects

References

Using Node.JS `--inspect` option