Node-RED Q & A on Much Ado About IT

Node-RED communications protocols

Julian Knight — Thu, 20 Aug 2020 17:53:20 +0000

Node-RED has an excellent choice of communications protocols available to it. This article looks at the main options and their strengths and weaknesses. It also gives suggestions as to when you may wish to use them.

Node-RED is renown for its MQTT support since that is where it started - as an IBM demonstrator for their experteese on IoT.

However, this is only one of many ways to send/receive data between Node-RED and other systems or services. Each of tese protocols has its own advantages and disadvantages.

The protocols

In rough order of descending order of ease of use (very rough!):

MQTT
Websocket
HTTP
TCP
UDP
UNIX Pipes

MQTT

This is likely to be the easiest to use even though it requires an extra service to be running, a “Broker”.

There is even a custom node available that lets you run a broker from within Node-RED itself. However, I don’t recommend doing that because it adds significant overheads that are easily avoided. The most commonly used broker is “Mosquito” which has low resource overheads, is fast, secure and capable. It is easily installed and configured.

While MQTT data payloads are just strings, Node-RED lets you send and receive more complex data structures (JSON).

The things that put MQTT at the top of the list are:

The ability to easily handle more complex data
The availability of MQTT libraries and clients for many platforms

Websocket

HTTP

TCP

UDP

UNIX Pipes

FAQ 4: How do I see the Node-RED log?

Julian Knight — Tue, 17 Mar 2020 16:28:07 +0000

Just as there are several possible ways to run Node-RED, there are several ways to access the log.

In line with most server type applications, Node-RED outputs all sorts of operational and error information to a log. In addition to Node-RED’s own logging, each node that you use may also add to the log.

You can also output your own information to the log from your flows, those methods are shown at the end of this article.

There are three categories of ways to actually run Node-RED and each will require you to access the log in a slightly different way.

Running Node-RED as a “Service”

If you install Node-RED using the script supplied for installation on a Raspberry Pi (or any other Debian-like host), Node-RED will be run as a system service. That is to say that it will be started up when the system boots. You can also do this on other platforms including Windows.

Linux

In these cases, typically system log output goes to a standard service such as syslog. Typically, the syslog output is found in a file at /var/log/syslog and you can use the tail command from a terminal command prompt to view it.

On most modern versions of Linux though, the syslog is being overtaken by logging built into the systemd service. In these cases, it is better to use the journalctl command to show the log output. This has the advantage that you only need to know the service name (node-red on a standard Pi install) and you don’t need to worry about where the actual data is stored. You can also use the more advanced features of that command to output logs in different formats.

The following command will show you the last 50 lines of the log, jumps to the last output and “follows” any further output.

journalctl -u node-red -e -f -n 50

Windows

Windows does not use log output in the same way that Linux does as it has the “Event Logs”. Typically, log output is redirected to a file and you will need to look at the startup command for Node-RED to see where the logs are being written to.

Running Node-RED manually

On all platforms, if you run Node-RED manually from the command line, the log output is shown in the same terminal. Using the keyboard command ctrl-c will both stop Node-RED and close the log.

Using a “runner” to run Node-RED

Sometimes, especially during development, it is helpful to use a “runner” application such as PM2 or nodemon to start Node-RED. These will recover from failures and can be made to “watch” for changes to files and restart automatically.

Each of these tools may have their own ways of viewing log output. For example, when using the popular pm2 runnner, the command is pm2 logs.

Changing log output levels

By default, Node-RED will show the following types of output from both Node-RED and any installed nodes:

fatal - only those errors which make the application unusable should be recorded
error - record errors which are deemed fatal for a particular request + fatal errors
warn - record problems which are non fatal + errors + fatal errors
info - record information about the general running of the application + warn + error + fatal errors

You can change the logging levels to also show:

debug - record information which is more verbose than info + info + warn + error + fatal errors
trace - record very detailed logging + debug + info + warn + error + fatal errors

Or you can remove levels or turn off logging alltogether (using the off setting).

In addition, there are 2 other settings that impact logs:

metrics - Whether or not to include metric events in the log output
audit - Whether or not to include audit events in the log output

All of these settings are found (complete with instructions) in the settings.js file typically found in the userDir folder (normally ~/.node-red/settings.js).

Outputting your own information to the log

When writing flows in Node-RED, you will normally use the debug node that, by default, outputs to the debug sidebar in the Node-RED Editor (admin UI). However, you can change the settings in the debug node to also (or instead) output to the log.

In addition, you can also output to the log from a function node using the code:

node.warn('some useful text')

Output:

Mar 17 17:12:48 pi node-red[18577]: 17 Mar 17:12:48 - [warn] [function:My useful fn node] some useful text

If you give your function node a reasonably name, you will see that in the output as shown. If the node doesn’t have a name, the id will be shown, you can use that in the Editor’s search feature to find the node.

FAQ 3: What different ways are there to install Node-RED and which is best?

Julian Knight — Tue, 10 Mar 2020 09:26:00 +0000

There are several possible ways to install Node-RED, this post provides an overview and some thoughts about when you might want to use which approach.

There are three basic approaches to installing Node-RED. Which of these is “best” for you will depend on your knowledge, willingness to learn, desire or need to gain maximum flexibility and so on. Only you can work that out. It should also be noted that you should be able to mix and max these approaches if you really want to (though I can’t say that I would personally recommend that).

Global Install

This is the standard and therefore arguably by far the easiest installation.
It is good for people who are not overly familiar with their OS’s filing system or how node.js and npm work and who don’t want to be bothered to find out (that is not a criticism by the way).
It allows for multiple instances to be run if needed but all instances must use the same version of Node-RED.
It doesn’t “pollute” your home folder with the node-red and many other required node.js modules.
It is standard and most tutorials and explanations assume that this is the way it is installed.
It makes the node-red command global so you can run it from anywhere.

Advantages

Best for beginners unless you are prepared to learn about Node.js.

Disadvantages

Less flexible and possibly a little less secure. Not the best for “production” installations. Can confuse beginners into installing other Nodes and packages into global when they shouldn’t. Hard to work out where the global files are.

Local Install

This is a more “node.js way” of installing - global installations being good for exposing global commands but may sometimes be less secure.
It is a more self-contained installation. Everything can be in 1 place under a single folder structure. This makes it easier to back-up everything together. But it doesn’t force you to work this way, you can keep your userDir folder separate if you like.
It allows you to run different instances of Node-RED using different versions. Good for testing major Node-RED upgrades and for testing custom nodes or flows against different versions.
You are in control. You know where everything is because you chose to put it there.
You control access to all files and folders and so this can be a more flexible installation. You could also have different instances of node-red on the same device with different folder/file access levels. Possible development and test instances on the same device.
A good way to create a more portable installation of Node-RED since the whole folder structure can be easily copied/archived, etc.
It is my personal belief that this is also the best approach for people who want to learn how Node.js and/or Node-RED works. But then again, maybe that’s just my warped mind!

Advantages

Best for flexibility and control. Good for control of security. Best for running different versions of Node-RED in parallel on one device. Good for ease of backup/recovery and for creating distributable copies of a Node-RED application. Best for embedding Node-RED into an existing or new Node.js app.

Disadvantages

Requires some knowledge of how Node.js and npm work.

Containerised Install

Uses either Docker or snap to containerise the installation.
Isolates Node-RED from the host OS.
Allow running multiple, independent instances of Node-RED, possible using different versions.

Advantages

Best for enterprise use where containerisation is already well supported.

Disadvantages

Adds significant hidden complexity and requires good knowledge of the container technology to isolate and fix issues. Also likely to have higher resource requirements than running separate instances in other ways. Experience from the Node-RED forum suggests that a decent knowledge of Docker/snap is recommended.

There is some additional information that I wrote in a recent blog post:

https://it.knightnet.org.uk/kb/nr-qa/faq2-multiple-instances/

FAQ 2: How can I run multiple instances of Node-RED on the same server device?

Julian Knight — Fri, 06 Mar 2020 01:21:33 +0000

There are all sorts of reasons to want to run more than a single instance of Node-RED and several ways to do it.

Use the command line settings to point to a different userDir folder.

node-red -u /data/userDirectory

That lets you use different flows and configuration settings but only allows a single version of Node-RED.

You still install Node-RED in the “standard” way, globally. While this is generally simpler for complete beginners, it can lead to complexities later on. You should understand the strengths and weaknesses of npm global installs.
Install the Node-RED npm package locally rather than globally.

This gives you not only the features from 1 but also allows you to run different versions of Node-RED in parallel. The alternate-node-red-installer shows you how to do that.

It also removes the need to install Node-RED globally which can lead to unexpected security issues.

This is also the “Node.JS way” and likely to be a more welcome approach for organisations and teams that run other Node.JS services and systems.

While slightly more complex to think about initially, especially for Node.JS beginners, it helps you understand how Node.js works, is easier to work out where everything is kept and is likely to be easier to maintain in the longer term.

I use npm’s script capability to make things really easy to start manually. Here are some examples. Note that I always make the userDir folder a sub-folder of the master project folder (where Node-RED itself is installed). Keeps things neat and makes it trivial to backup the whole thing.
```
{
 ...
 "scripts": {
 "start": "node node_modules/node-red/red.js --userDir ./data",
 "start2": "set DEBUG=express:* & nodemon node_modules/node-red/red.js --userDir ./data",
 "inspect": "node --inspect node_modules/node-red/red.js --userDir ./data",
 "log": "sudo journalctl -u node-red -f -n 0 -o cat",
 "update": "npm install --unsafe-perm --production node-red",
 },
 ...
}
```
So you can start manually using npm start and start with debugging port active using npm run inspect
Run Node-RED in a container such as the Docker container.

Allows multiple versions of Node-RED along with different configurations and supporting software and services.

However, this comes at the cost of significantly greater complexity and resource overheads. Unlikely to be worth doing except in specific circumstances & needs a reasonable level of understanding of how Docker works.

You will realise that I much prefer method 2 for all of my own installations. That’s why I wrote the alternate installer which goes some way towards making installations easier and more consistent.

For running Node-RED on system startup, please see the Starting Node-RED on boot page in the docs.

FAQ 1: Should I use a function node or a flow of core nodes?

Julian Knight — Fri, 06 Mar 2020 01:16:35 +0000

There are usually many ways to achieve something in Node-RED. A common question is whether or when to use a function node or a collection of core nodes to achieve an outcome.

Whether I use one approach or the other depends on the complexity of the input vs the complexity of the output. It really doesn’t matter except that it is often easier to read a single function node that several complex change nodes as an example.

So my general rules of thumb for whether to replace a collection of nodes with a function node are:

How difficult would this be to do in a pure flow? If it is just a couple of nodes, that’s fine, if it is 10 nodes then probably not fine.
If I am using JSONata, how long is it taking me to puzzle over the JSONata code? If it is a few minutes then fine, otherwise use a function node!
Did someone already write some JavaScript that I can purloin? If so then use it!

Of course, I know JavaScript reasonably well and if you don’t then your own rules might look a little different.

A Glossary of Node-RED terms

Julian Knight — Fri, 21 Jun 2019 17:40:30 +0000

For those not used to it, the terminology used in Node-RED can occasionally be confusing. Due in part to the overloading of some terms. So I've come up with this glossary to help keep myself from getting confused.

A draft set of terminology for Node-RED, the flow-based visual programming tool.

Glossary

Editor: The Node-RED admin UI. Where you edit and deploy flows.
Flows: The collection of node instances and wires defined by the Flows file loaded by Node-RED at run time.
Flow: A tab in the Node-RED admin UI used to break Flows into more manageable chunks.
flow: A set of node instances and wires with in a Flow tab.
Package: An npm package used, amongst other things, to install Node-RED Nodes.
Node: The source definition for nodes that can be used in Node-RED Flows. Defined normally by an npm Package (though actually defined by 3 files: package.json, an html file for the admin UI and a JavaScript file for the server process).
Node Instance: An instance of a Node added to a flow. A Node (definition) may have many instances in Flows.
node(s): same as Node Instance(s).
wire(s): The logical connectors between nodes. Conceptually, messages “travel” between nodes. In reality, they don’t, data is simply passed by reference (except for the edge cases where a message copy is forced).
message(s): Often referred to by the short name msg which is the variable name generally used to reference the message. This is the data object used to communicate between nodes. Typically, it has the minimal definition: {{_msgid: uid, topic: undefined|string, payload: Object|Array|string|number}} but other than requiring _msgid (which is added by Node-RED), it may contain any valid, serialisable JavaScript.
userDir: The server folder that contains the Flows file, settings.js and other Node-RED configuration information. Typically residing at ~/.node-red.
Project: a Flows file and other configuration files that allow rapid switching between Flows. Note that a project shares the same set of Packages (and therefore Nodes) and global Node-RED settings as all other projects. Only the Flows file, matching credentials, a README and a minimalist package.json are part of a project. Projects are designed to enable teams to work together and can use GIT to manage the data collaboration.

Creating a notification in the Admin UI

Julian Knight — Sun, 16 Jun 2019 19:22:16 +0000

Node-RED's admin UI has a built-in notification (toast) pop-up message feature. This article explains how to use it when creating custom nodes as it isn't currently well documented.

RED.notify('Your message here', options)

This can be called from the script in your admin ui code <nodeName>.html.

By default, the notification will disappear after a few seconds.

Options

type: {success|warning|error=} Adds style to the message box.
fixed: {boolean=false} Defaults to false, the notification will auto-remove after a few seconds.
modal: {boolean=false} If true, no other user interactions are allowed until the notification has cleared.
timeout: {ms=5000}: Specify the time in milliseconds a non-fixed notification will be displayed for.
id: {string=} Allows a notification to be updated.
width: {px=} Desired width in pixels. (Will be limited to the parent width)

Methods

.close(): Close the notification (useful if fixed option is true).
update
hideNotification
showNotification

Example

let myNotification = RED.notify('My message in a box', {
 modal: true,
 fixed: true,
 type: 'warning',
 buttons: [
 {
 'text': 'cancel',
 'click': function(e) {
 myNotification.close()
 }
 },
 {
 'text': 'okay',
 'class': 'primary',
 'click': function(e) {
 myNotification.close()
 },
 },
 ],
})

References

https://discourse.nodered.org/t/node-red-admin-ui-display-message-to-user/12290
https://github.com/node-red/node-red/blob/ccc3809daa3b03e7171cb6f4ccf4d6e0f1188bef/packages/node_modules/%40node-red/editor-client/src/js/ui/notifications.js#L16
https://github.com/node-red/node-red/wiki/API-Reference (old but has some things not documented elsewhere)

Cleaning up npm package install problems

Julian Knight — Wed, 05 Jun 2019 07:05:39 +0000

Node-RED uses npm to manage package installations such as its nodes. Sometimes, you can get into a situation where you start getting lots of npm installation or removal issues and it feels like you are going round in circles trying to fix things. If so, time to call it a day and to get drastic! Here, I try to explain an easy way to fix things quickly and completely without side effects.

Node-RED is built over Node.js which uses npm as its library (AKA package) manager. It is great to have a standard mechanism to manage libraries but the complexities of managing JavaScript based packages sometimes trips things up.

Once you’ve tried for a bit to understand what the issues are, it may be time to call time and get drastic.

The following commmands will clean out the packages installed for your live instance of Node-RED - assuming you have it installed via the “standard” default method. If not, you will need to adjust the folder you operate in.

cd ~/.node-red
rm package-lock.json
rm -R node_modules
npm install

That will cleanly reinstall all of the packages marked in your package.json file. So make sure that is correct first. Every time you install a package (as long as you have reasonably up-to-date versions of Node.js and npm) whether from the NR admin ui or command line, it gets added to package.json.

I now always remove the lock file as well if I’m having issues as that can be the cause of some issues. The lock file isn’t generally needed in NR installations.

Note, as always, I’ve assumed you have a “standard” install of Node-RED which puts your userDir as ~/.node-red (and the equivalent on Windows). If you use an installation scheme like mine, it will be in a different place as noted by the command used to start NR.

Analysing the Performance of Node-RED

Julian Knight — Fri, 19 Apr 2019 13:23:06 +0000

Node-RED is a powerful platform for prototyping and rapid development but it is complex. It has a lot of moving parts and it can be hard to know what might be causing performance bottlenecks. This article tries to give some hints and tips about analysing those bottlenecks and how to resolve them.

Sorry, this is draft and far from complete. Please leave comments if you have something useful to add. Thought I would publish anyway in case anyone finds it useful. I will try to come back to it in the future. If you leave a comment, it will remind me. :-)

Platform

Before starting more detailed analysis, the first thing to do is to look at the platform being used to run Node-RED.

Often, something like a Raspberry Pi or other single-board computer (SBC) is being used. They have very limited resources and often slow storage (e.g. SD-Cards). So it is very easy to overload them and kill performance.

To analyse this, you need to use a tool such as top. Better still, get something a bit more comprehensive such as glances. Either way, these will tell you how much CPU and memory (RAM) is being used and by what. Start by looking at the three ’load’ values, these should all be fairly low, if they are going over about 2-3 regularly then you have an issue.

Next look at the ‘swap’. If this is being used regularly, you will have a performance bottleneck because memory (RAM) is being overused. To fix this, you have to unload some services. If using the glances tool, it will show a warning if the memory is swapping.

The easiest way to remove load is likely to be turning off the ‘desktop’. This gives you a GUI that you directly interact with via keyboard and monitor. Removing this will remove lots of load on memory and CPU. Have a look at articles that explain how to run your device ‘headless’. You will need to be able to connect to your device over the network from another device.

After that, if there are still issues, you need to look at taking off services that you don’t actually need running.

Finally, if you still have issues, you will need to think about splitting services between multiple devices or using a more powerful device.

Back End (server)

Using Node.JS `--inspect` option

Front End Browser (Node-RED Admin UI)

For any browser performance issues, you firstly need to ensure that the problem is actually coming from something related to Node-RED. So you should turn off any browser extensions before continuing. An extension such as SimpleExtManager for Chromium browsers will assist with easily turning on/off extensions.

Browser Performance Profiling

Using your browser’s developer tools, you are able to record and analyse a performance analysis. The Chromium developer tools have three tabs to help with this. Performance, Memory and Application.

The Performance tab is perhaps the easiest to get going with. Click on that tab and then start profiling. Leave that for an amount of time that you estimate will let you see any ongoing performance issues. That might be just a few seconds or a few minutes. You will want to tick the “memory” option and probably don’t need the “Screenshots” option.

The thing to look at first is likely to be the memory utilisation. There is a graph 1/2 way down the display that shows you some important metrics including the “JS Heap”. See how big that is getting by hovering over the line. Also look at how fast it is growing. You should see that the heap reduces periodically as the JavaScript engine does a ‘garbage collection’.

You can also do multiple recordings and compare the heap over a longer period.

Front End (Dashboard, uibuilder, http-in/-out nodes, etc.)

A more comfortable way to edit the code for creating custom nodes

Julian Knight — Mon, 18 Mar 2019 17:38:40 +0000

Node-RED allows us to create custom nodes to extend the features. While this is a great feature, actually writing the code can be tricky. In part because the code for the admin configuration panel is contained in an html file containing 3 script entries that are Node-RED specific. This post shows you a relatively easy way to split that into 3 files and combine them with a build step using an npm cross-platform script.

When defining a new node in Node-RED, the code is contained in two files. An html file and a JavaScript file.

The JavaScript file contains the core code that is loaded into the server and does most of the hard work. As this is a standard js file, nothing special is needed to edit it, your favourite code editor or IDE will do the job nicely. My preference is VScode from Microsoft.

However, the html file isn’t very comfortable to edit because any HTML it actually contains is wrapped in some special <script> tags that are only recognised by Node-RED and they throw out your code editor making things really hard to read and process.

<script type="text/javascript">
... js that is run in the admin ui when the configuration panel is opened ...
</script>

<script type="text/x-red" data-template-name="uibuilder">
... html that is used in the admin node configuration panel ...
</script>

<script type="text/x-red" data-help-name="uibuilder">
... html that is used in the information sidebar ...
</script>

To get round this, I’ve started creating a separate ./node-src folder and splitting the content for this file into three source files: script.js, template.html and help.html.

Of course, you can call them whatever you like & will probably want to include the node-name if you are creating multiple nodes.

There is then a cross-platform executable that replaces the empty master with the contents of the three files.

Now all you have to do is to remember to run the build before pushing your changes to your repo and before publishing a new version to npm. That can also be automated as long as you are using npm to manage git. Look at the prepublishOnly script which will automatically allow you to run the build before publishing. You can do similar things with git by setting up the appropriate scripts.

Installation

To get this all working, you will need to firstly install a package called replace-in-file. Install this to the root of the folder holding you node.

npm install replace-in-file -D

Note that I install it as a devDependency because it is only needed for the node’s editors and not for end users.

Next, add the following to the package.json file, also in the root folder of your node.

 ...
 "scripts": {
 "build": "node ./bin/mergehtml",
 "prepublishOnly": "npm run build"
 },
 "bin": {
 "mergehtml": "./bin/mergehtml"
 }
 ...

This lets you run the build on any platform supported by npm, including Windows:

npm run build

Now, create folders ./bin and ./node-src. Into node-src, put the three files and your master template. Into bin put the mergehtml.js file shown below.

./bin/mergehtml.js

This is the code that does the magic. I overwrites the nodes html file (./nodes/node.html in this example) with the blank template then replaces the three <script> tags with the contents of the files from node-src.

#!/usr/bin/env node

/** Merge the various sections of nodes/uibuilder.html from the files in node-src/
 * Note that this is run from `npm run mergehtml` which puts the working directory
 * to the root, not `./bin`
*/

const replace = require('replace-in-file')
const fs = require('fs')
//const path = require('path')

// nodes\lib\uibuilder-help.html
const myhelp = fs.readFileSync('./node-src/help.html')
const mytemplate = fs.readFileSync('./node-src/template.html')
const myscript = fs.readFileSync('./node-src/script.js')

// Copy template - overwrite the nodes html file
fs.copyFileSync('./node-src/node.html', './nodes/node.html')

var options = {
 files: './nodes/uibuilder.html',
 from: [
 /(<script type="text\/javascript">)(.|\n)*?(<\/script>)/gmi,
 /(<script type="text\/x-red" data-template-name="uibuilder">)(.|\n)*?(<\/script>)/gmi,
 /(<script type="text\/x-red" data-help-name="uibuilder">)(.|\n)*?(<\/script>)/gmi,
 ],
 to: [
 `$1\n${myscript}\n$3`,
 `$1\n${mytemplate}\n$3`,
 `$1\n${myhelp}\n$3`,
 ],
}

replace(options)
 .then(changes => {
 console.log('MERGEHELP: data-help-name Modified files:', changes.join(', '));
 })
 .catch(error => {
 console.error('MERGEHELP: Error occurred:', error);
 })

console.log('MERGEHELP: Completed')

./node-src/node.html

Here is the content of the node.html file - you can use this for any project. You can put text between the tags if you like, even change the order of them if you prefer, the code should still work just fine. Obviously, any text you put between the open/close tags will be replaced.

<script type="text/javascript"></script>

<script type="text/x-red" data-template-name="uibuilder"></script>

<script type="text/x-red" data-help-name="uibuilder"></script>

How to create secure certificates

Julian Knight — Sun, 14 Oct 2018 15:54:20 +0000

Generate certificates for Node-RED that are trusted by all modern browsers. This will let you access Node-RED (and other services) over an encrypted HTTPS link.

THIS ARTICLE IS CURRENTLY IN DRAFT - it is still being developed. Please feel free to add constructive comments and corrections below.

The problem

Argh! Why is it so hard to create and manage trusted certificates for “internal” services! While you can create your own “self-signed” certificates, all modern browsers now mark these as insecure and try to stop you from accessing them. This is wrong. Browsers should allow access to self-signed certificates if they point to a non-routable IP address (192.168.. or 10...* for example) or an invalid root domain such as *.something.local.

Also, we really probably don’t want to expose all of our internal servers to the bad, wide Internet - this is generally a really bad idea unless you are good at securing things and have the time to keep making sure they stay secure as things update.

Possible fixes

You could manually add a new (self-signed) root certificate to all devices needing access to your internal services so that your self-signed certificates are trusted - try getting that past the rest of the family!

The only other alternative is to use a trusted CA. Since I’m assuming you are doing this for testing or for use at home, I also assume that you don’t want to spend lots of money. Trusted certificates usually cost - a lot! Often US$100 per year or more.

However, there is one supplier that issues free trusted certificates. Let’s Encrypt. This is a great service for a great price. But it comes with some overheads.

It’s never simple

OK, so assuming we want to use Let’s Encrypt (LE), what issues do we now need to overcome?

Firstly, we have to have a publicly known domain address. You cannot issue a publicly trusted certificate to an IP address or non-routable domain name.

Note that certificates are generally issued to specific domain names so that www.thing.com and thing.com are different names. We don’t want to mess with all of that all the time, especially if we are doing lots of tests. So we can now use a “wildcard” certificate for *.thing.com
Next, we have to have a way for the Let’s Encrypt servers to verify that our domain name is actually ours to do something with.

By default, the LE service wants to have access back to our server in order to verify that it is ours. This would bring us back to the issue of exposing our server to the Internet, something that we prefer not to do unless we really have to.

Thankfully LE now has an alternative called DNS-01. Unfortunately, this requires our DNS (Domain Name Service) to support a particular type of secure API. For that we can use Cloudflare or any of the other DNS services listed on the LE website.
Finally, we have to renew the certificate every 3 months since that is all that Let’s Encrypt allows us to keep a certificate for. This is a pain but it does have some security benefits.

Let’s look at the details of how to do this.

1. Get a domain

We need a domain and we need it to be one that we can, at least to some degree, control. So some of the free “dynamic DNS” services probably won’t cut it. However, we can get our very own domain for a few £/$ per year so go ahead and do that. Save yourself some pain in the next bit by using Cloudflare themselves to register your domain.

Once we have a domain, we need to let it be managed by a DNS that supports the DNS-01 verification API. You might be lucky in that your domain registrar already supports that. In most cases, they won’t. So now you have to hand over control of the DNS settings to someone like Cloudflare who’s free service is plenty for what we need. Your domain registrar will tell you how to change the name servers that control your domain. Don’t worry, your registrar remains the overall controller so renewals are not an issue.

As we will be using a wildcard certificate, we don’t need to worry about setting up specific names at this point.

Also, as we will be using DNS-01 verification, we don’t need to point anything at our public IP address. Use the DNS settings to point the default “A” DNS entry at a dummy IP address like 10.10.10.10. That’s fine and it means that we won’t be leaking any information about our private network.

2a. Install a client for Let’s Encrypt

There are lots of clients for LE, see the list on their website. For this post, I’m going to use a Raspberry Pi as I have one running permanently controlling my home automation system. You could also use a NAS or any PC. Even some routers such as the ever excellent Ubiquity EdgeRouter Lite can be used.

I am going to use a 3rd-party BASH (Linux command-line) script as this is a lot simpler than the official Python based script.

From your Pi, follow the instructions to install the script. Start a remote command line using an SSH client from a convenient computer. Log in using an ID that is allowed to do administration on the Pi. Then install the script using curl https://get.acme.sh | sh. Note that I didn’t bother to become root as I need to use the certificate with a Node.JS service that isn’t run globally as root (this is best practice, don’t run things as root as that opens up additional security issues). I also ran alias acme.sh=~/.acme.sh/acme.sh manually rather than logging out and back in again to pick up the defined alias.

2b. Get the Cloudflare API details

So, we’re ready now right? Not quite. First we need somewhere to actually run the client tool that will initially get our first certificate and that will then run periodically to renew the certificate.

As we are using DNS-01 validation with Cloudflare, we need the API access details. Jump ahead in the acme.sh instructions to the part on DNS API integration since that’s what we need in order to avoid exposing our internal servers to the Internet. We need a configuration file. The instructions are in the the dnsapi folder.

Log into your Cloudflare account, go to “My Profile” under the little person icon top-right. Pick up your verified email address then scroll down to the bottom “API Keys” & click on “View” against the “Global API Key”. While you are there, turn on 2-factor authentication to protect your account and services. Keep this information safe! If someone gets hold of it, they can change your DNS and other settings.

Issue the commands export CF_Key="sdfsdfsdfljlbjkljlkjsdfoiwje" && export CF_Email="xxxx@sss.com" which temporarily puts the security information into environment variables.

2c. Get our first certificate

Now we are ready to try and get our first certificate. Run the following from your SSH command line:

acme.sh --issue --dns dns_cf -d example.com -d *.example.com

Obviously replacing the domain names with your own. The entry that starts with * gives you a wildcard certificate so that you can use this certificate with any sub-domain like www.example.com or fred.example.com.

3. Certificate renewals

As mentioned, LE certificates expire every 90 days. Thankfully, the script we just ran not only does gets us our first certificate, it sets up a script to renew the certificate every 60 days - giving some extra time for the occasional renewal failures. You can adjust the renewal in the configuration file if you really want to. You don’t have to worry about restarting this if your device reboots.

Once the script has run for the first time, go ahead and run the following so that the script itself auto-updates:

acme.sh --upgrade --auto-upgrade

Using names instead of IP addresses

New we have a certificate that we can use for any service simply by referencing that service by name.

Note, though, that you can no longer reference your services by IP address.

For example pi.example.com but not 192.168.1.20.

Using an IP address will throw an error in modern browsers.

So a slight wrinkle in our effort to get rid of browser errors since, most home servers are access via an IP address not a name. How do we fix that? We have a couple of options.

Use your router’s DHCP or DNS service to define names for local services.
Install a DNS service on a local device.
Update the hosts file on every client device.

This may work if you only have one or two laptops and nothing else but even then it is clunky. With mobile devices, it won’t be possible anyway.

Both 1 and 2 will require some configuration. What we are doing is creating names that point to local IP addresses. How you do this, depends on the router or DNS server you have.

In either case, we really want a fixed IP address to work with so that our Pi (or other device running our services) is always at the same address. All routers should have the ability to do this so look for the DHCP settings. You will need to know the MAC address of the device, it looks something like b8:27:eb:df:49:7e (you may see it with upper case letters and/or without the colons). The router’s DHCP pages will have a list of active “leases” that will show that information. You use that address, which is defined by a network interface on a device, to issue a fixed IP address via the DHCP service. Make sure that your fixed addresses don’t overlap with the range defined for DHCP to issue dynamically.

Sometimes, your DHCP server will let you define a name to go with this configuration. That’s great because you can now specify the name to include the domain that the certificate is issued to. e.g. pi.example.com.

The Ubiquiti EdgeRouter’s will let you define names manually using a Wizard called “DNS Host Names”. Some other routers may give you access to the router’s hosts file.

If you can’t do that on your router, you will need a DNS server that will let you define names to IP addresses. That’s a bit more complex and beyond the scope of this, already rather long, blog post I’m afraid.

Using your certificate

Now you have the certificate and can access your servers via names instead of IP addresses so you are ready - finally - to configure the services to use the certificates.

You can configure most TCP/IP based services to use TLS (Transport Layer Security) which is what we will mostly want our certificate for. Most people will be familiar with accessing web pages over HTTPS which is TLS applied to HTTP. But we can also use the certificate to secure communications for file transfers (FTPS or SFTP), Email (SMTPS, IMAPS, etc.).

Node-RED

Node-RED is a service built over NodeJS and ExpressJS. It creates a web server that we can secure using our certificate. The same certificate will also be used to help to secure websocket communications.

Once you’ve changed the settings below, remember to access Node-RED using the server name instead of the IP address. You will need to restart the Node-RED service.

settings.js

This file is where we configure Node-RED to use HTTPS. Note that the settings are the same as those from NodeJS so you can check out any other settings in the NodeJS documentation.

The settings.js file is found in your userDir folder which is generally ~/.node-red if installed according to the instructions on the Node-RED website. ~ is the “home” folder for the user ID running the Node-RED service.

...
module.exports = {
 ...
 const path = require('path');
 const fs = require('fs');
 ...
 https: {
 // Don't forget to adjust the paths below
 // according to your installation
 key: fs.readFileSync(
 path.join(
 '..','.acme.sh','<server_name>', '<server_name>.key'
 )
 ),
 cert: fs.readFileSync(
 path.join(
 '..','.acme.sh','<server_name>', 'fullchain.cer'
 )
 ),
 },
 ...
}

Where <server_name> is something like pi2.example.com - whatever you have defined as the name associated with the IP address.

MQTT broker Mosquitto

NOTE: This section is not complete as I’ve not managed to get this working as yet.

Many IoT systems make use of MQTT for publish and subscribe handling of data from devices. The more we embed IoT into our lives, the more important it is to secure the MQTT brokers. By enabling encrypted communications via TLS and then configuring user ID’s and strong passcodes between devices and the broker, we can help make things a lot more secure.

Mosquitto is one of the most common MQTT brokers due to its small size and high performance.

On a Linux system, Mosquitto configuration files are found in /etc/mosquitto. Try not to edit /etc/mosquitto/mosquitto.conf, instead add your own file to /etc/mosquitto/conf.d. It will be loaded automatically and will not be overridden when Mosquitto is upgraded.

/etc/mosquitto/conf.d/custom.conf

Webmin

Webmin is a really helpful remote administration tool for Linux servers. Being web-based means that you can use your certificate with it. Replace the default certificate by going to Webmin > Webmin Configuration > SSL Settings (https://<domain_name>:10000/webmin/edit_ssl.cgi?xnavigation=1).

Then change the default Private key file path with the the one generated by the acme.sh script (it ends with .key). Also change the certificate file setting to be the file fullchain.cer in the same folder.

Now restart Webmin from the main Webmin configuration page or issue the command sudo systemctl restart webmin.

No more certificate errors as long as you remember to use your server name not IP address.

Web servers

There are so many posts about configuring any of the regular web servers with certificates that I’m not going to repeat them here.

Just remember that you have all the certificate and key files that you need so you only need to make the folder available (read-only) to the user ID that runs the web server. Then to configure the appropriate files.

Router (Ubiquiti EdgeRouter)

There is a dedicated script for EdgeOS which you may wish to use for simplicity. Otherwise, you will need to remember to securely transfer the fullchain.cer and xxx.key files to the router’s filing system each time they are updated.

You can change to custom key and certificate files using the “Config Tree” service / gui.

InfluxDB and Telegraf

InfluxDB is a timeseries database that is very efficient at recording data over time. Great for sensor data. All access is, by default, over HTTP so it is possible to configure it to use HTTPS.

Telegraf is from the same vendor as InfluxDB. It can also be configured to talk to InfluxDB over HTTPS.

Grafana

NOTE: Grafana seems to want to be able to write to something - either the folder or the cert/key files - that it shouldn’t do so currently is failing. I haven’t had time to resolve this as yet.

/etc/grafana/grafana.ini

[server]
# Protocol (http, https, socket)
protocol = https

# https certs & key file
cert_file = /home/pi/.acme.sh/<server_name>/fullchain.cer
cert_key = /home/pi/.acme.sh/<server_name>/<server_name>.key

How to secure Node-RED

Julian Knight — Sun, 30 Sep 2018 13:48:14 +0000

Node-RED is increasingly used in situations that require reasonable security. Up to now, however, the information required to secure it correctly has been fragmented. This article aims to give an end-to-end outline to enable anyone to secure their installation.

THIS ARTICLE IS CURRENTLY IN DRAFT - it is still being developed. Please feel free to add constructive comments and corrections below.

I am going to attempt to describe the various options and configurations that you will need to think about in order to secure an instance of Node-RED. This is probably going to be a long article!

Please see my other articles Making Node-RED available over the Internet and Secure Home Automation Controls via a Telegram Bot for other ideas about the security of Node-RED in regard to use over the Internet and how to avoid having to worry about some of the issues dealt with here.

Warning and disclaimer

This is my best view of securing Node-RED. I’ve not been a professional developer in a long time and I am not a professional security analyst (I’m a IT technology and information security manager).

So you must not assume that this article covers every issue. You must also not assume, even if you follow every best practice, that an instance of Node-RED will be “Secure”. It may not be for many reasons.

Get your system and its infrastructure tested to destruction by professionals.

Hopefully though, this article will at least let you see the issues and have a reasonable go at making your instance of Node-RED a little more secure.

Terminology

A few words and phrases that will be used along with the least technical descriptions that I can come up with.

Term	Meaning
Encryption	Changing human readable information into something that requires a special key before it can be understood.
Authentication	Providing a secured identifier to a system to prove that the person accessing the system is who they claim to be.
Authorisation	Controlling what information a user of a system is allowed to see and what actions they are allowed to take.
TLS	A defined protocol for securing connections between systems and applications. In basic use, it provides a minimum level of authentication of a server (via a certificate chain of trust) and facilitates encrypted communications with that server.It also has additional mechanisms that will also provide a minimum level of trust of the client application. However, this is rarely used in Internet web applications.TLS may be used to secure many different application and system interactions, not just between a browser and a server.
HTTP(S)	The main web protocol that lets browsers get human readable information from web servers. HTTPS is HTTP secured with TLS (used to also allow a protocol called SSL but that is deprecated as insecure). Ideally, every connection to a server from a browser should be over HTTPS since the use of HTTP allows some pretty bad things to happen.

The basic architecture of Node-RED

To help us understand what needs to be secured, here is the basic technical architecture of Node-RED. From the outside in. This is only one way to view the architecture of course, I’ve tried to keep things as simple as possible.

Your flows, Dashboard and other pages served by Node-RED

This is the “code” that you (or your users/administrators) have put together.

Node-RED is a general purpose tool and so is capable of creating web pages and various other types of connections such as websockets, TCP/UDP connections and much more.

Anything that is created at this level either must be secured by Node-RED’s settings or by your code.
The Node-RED runtime API

This is the bit of magic that makes everything work. At some point, this will be independent to the administration side but at the moment (~ v0.19) it isn’t.
The Node-RED Administration user interface

This is a web application (web page plus websocket connections) that lets you build your own applications using Node-RED. It uses various 3rd-party libraries to do some of the heavy lifting (D3, JQuery, etc.)
The Node-RED core and contributed nodes

Here is all of the clever custom code, generally tucked out of the way so you don’t need to worry about it.
ExpressJS

ExpressJS is a library for NodeJS that does the heavy lifting of providing a web server.
NodeJS

NodeJS is a collection of JavaScript and C code that lets people build application servers across different operating systems. It lets people use JavaScript (typically a browser language) to create their server applications.
Operating System

Apart from the hardware, we are now at the bottom of the architecture.

No point in securing everything else unless you have secured things at this level.

In order to secure Node-RED, we have to pay some attention to all of these layers. Thankfully, the guys to created Node-RED have thought about many of the details and made it at least somewhat easier to provide a basic level of security.

Settings that you need to think about

Other considerations

NodeJS and package vulnerabilities

As Node-RED itself needs to have backwards compatibility with older versions of NodeJS, it may be forced to use packages (AKA modules or libraries) that are now out-of-date. Similarly, nodes that you rely on may not get updated as regularly as needed to stay ahead of vulnerabilities.

NodeJS itself is very rapidly changing and unless you are updating it regularly (e.g. weekly) it may also have outstanding vulnerabilities.

To mitigate these issues, you need to:

Update NodeJS weekly or daily in sensitive installations. You will also want a set of regression tests - test that are run after every update - if you want users to be able to rely on rapid update cycles.
npm itself may have vulnerabilities and may need to be updated out of cycle from NodeJS.
Regularly run npm outdated and npm audit on both the location you install Node-RED (global by default) and the location where you install contributed nodes. Update nodes regularly or consider the impact of nodes that have dependencies failing audits. Note that npm audit is only available on newer versions of npm.

Issues and limitations

Specifics

References

Node-RED Official documentation on security
Making Node-RED available over the Internet
Secure Home Automation Controls via a Telegram Bot
Why should we use HTTPS?
1. Google’s take
2. Cloudflare’s take
3. The US Government’s take
4. A leading security experts take (Troy Hunt)

Drayton Wiser heating control

Julian Knight — Mon, 13 Aug 2018 17:31:52 +0000

Drayton are a very widely used manufacturer of heating controls. They make the "Wiser" control system which is low cost and easily fitted. This article shows you how to use Node-RED to query and control the system.

Drayton’s Wiser smart heating controller is a low-cost system for controlling boilers and radiators. Note that Drayton is part of Schneider-Electric

It has the lowest cost thermostatic radiator valves (TRV’s, smart radiator controls) of any of the smart home systems - around half the price of other systems. The control unit is also low-cost.

It also has the advantage of not requiring the Internet or the Drayton cloud servers for the system to keep running so you are not dependent on Drayton keeping the Wiser cloud service active. This was a really important point for me when choosing a smart heating controller since it is all too easy to end up with a “brick” instead of a working system.

However, the disadvantage of the Wiser system is that it isn’t as sophisticated as some others and clearly Drayton don’t have the resources to rapidly develop it as some other manufacturers have (such as Honeywell for example).

Of course, if you are reading this, you are probably already somewhat knowledgeable about home automation. You may also be aware of Node-RED which is a tool well suited for creating custom home automation systems with a minimum of knowledge and little to no coding required.

We can easily use Node-RED to help get information from the Wiser system and even use it to add smarter controls such as connecting to other sensors or to other systems and services.

At the end of this article is some code that you can import into your own Node-RED service and start working with the Wiser system straight away.

Connecting to your controller

You will need to get a couple of pieces of information before you can start working with your Wiser system and Node-RED.

Once you have this information, you can configure the example Node-RED flow at the end of this article and you should be able to query information and override some settings at will. Once you have mastered the basics, you will be able to do pretty much anything you want to using the power of Node-RED.

IP Address

When your Wiser system connects to your home Wi-Fi, it will be given an IP address and we need to know that before we can talk to it. There is no really easy way to do this. The address will generally be set by your Internet router (or Wi-Fi access point) so you should start by checking your router’s web interface for “DHCP”. Hopefully you will see a list of assigned DHCP IP addresses, the Wiser controller is likely to be the highest number if you have just connected it.

You should configure your router to give the controller a fixed IP address otherwise it may change if the router is restarted.

System Secret

In order to interface with the controller, you need to know a very long string of random characters that is its system secret.

You can get the system secret by doing the following (reference):-

Press the setup button on your HeatHub, the light will start flashing
Look for the Wi-Fi network (SSID) called ‘WiserHeatXXX’ where XXX is random
Connect to the network from a Windows PC
Once connected run the following command in PowerShell

Invoke-RestMethod -Method Get -UseBasicParsing -Uri http://192.168.8.1/secret/

If you are using a different type of operating system, you will need a tool that will return data from a web query.
This will return a string which is your system secret
Press the setup button on the HeatHub again and it will go back to normal operations
Copy the secret and save it somewhere.

Node-RED flow to configure IP address and System Secret

To save having to repeatedly enter the IP address and system secret into your flows, use the following flow to put them into memory. The query and control example flows will use this later on. This flow will run automatically every time you start Node-RED.

Don’t forget to change the change node (the 2nd node in this flow) to the information you obtained above.

Example Wiser Configuration Flow

[{"id":"738ed75a.9b0dc8","type":"inject","z":"fb0c842.c87fc78","name":"Save controller configuration for Wiser",
"topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":true,"onceDelay":0.1,"x":230,"y":460,
"wires":[["d34f3b88.6b3008"]]},{"id":"d34f3b88.6b3008","type":"change","z":"fb0c842.c87fc78",
"name":"Set Wiser Variables (Secret and Host)","rules":[{"t":"set","p":"wiserSecret","pt":"flow",
"to":"<Change_this_to_your_systems_secret_string>","tot":"str"},{"t":"set","p":"wiserHost","pt":"flow","to":"<your_controllers_ip_address>",
"tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":460,"wires":[[]]}]

Copy the code in the black box and use the “Import” menu in Node-RED.

Controlling the system

The following sections contain basic information about how to control the Wiser system.

Don’t forget that the system may be controlled by other applications (like the mobile app) or services such as Google Home or Amazon Alexa integrations.

Note that you do not need the Internet for this at all. You only need your local network. All of this will work even if your Internet connection is down or the Wiser cloud service stops working.

Each entry below has a “path” such as ‘/data/domain/System/RequestOverride’ and some text or JSON in a grey box. In the example flows, the information in the grey box goes into the payload property (msg.payload) and the the path goes into the path property (msg.path). These property values are then used by the flow and sent to the controller. These property values are set in the node that follows the inject node, they are the second node in each flow, they are called “change” nodes.

Room ID’s

The Wiser system mainly works by controlling “rooms”. In the mobile app, these are all named and you can find the names in the data you can query as well. However, in order to control the system from Node-RED or any other home automation system, we need to know the ID of the room.

Wherever you see <roomNumber> in the information below, this is the ID not the offset in the /data/domain/Room array.

Temperature values

The values use to control and query temperatures are multiplied by 10. So 200 equals 20.0°C, 195 equals 19.5°C. Note that the value will be rounded to the nearest ½ a degree by the controller.

Set or cancel an override of the thermostatic temperature for a Room

Path: /data/domain/Room/<roomNumber>

Payload: {"RequestOverride":{"Type":"Manual","SetPoint":195}}

This will be permanent until something else changes the setting.

Set/cancel Boost for a Room

/data/domain/Room/<roomNumber>

{"RequestOverride":{"Type":"Manual","DurationMinutes": 30, "SetPoint":200, "Originator":"App"}}

{"RequestOverride":{"Type":"None","DurationMinutes": 0, "SetPoint":0, "Originator":"App"}}

Note: Duration is in minutes. The returned info includes a OverrideTimeoutUnixTime property, to convert this to a real time, you can use the Unix timestamp converter website.

Set/Cancel Manual mode for a room

/data/domain/Room/<roomNumber>

{"Mode":"Manual"} or {"Mode":"Auto"}

Set/Cancel window state detection for a room

/data/domain/Room/<roomNumber>/WindowDetectionActive

true or false

Set/cancel Away Mode

/data/domain/System/RequestOverride

{"type":2,"setPoint":100}

{"type":0,"setPoint":0}

Note: Sets the min temperature to 10°C

Also note that you should cancel manual overrides for all rooms.

References

Much of the above was gleaned from the following reference code.

andrew-schofield/openhab2-addons - also has references for hot water, scheduling and smart plugs
chrisduffer/drayton-wiser

Example control and query code

Copy the code and import it into Node-RED. You will need the configuration flow as well.

The configuration flow sets the flow variables flow.wiserHost and flow.wiserSecret.

Query the current settings

Query everything or query a single room

The top inject will return everything that the controller can tell us. It is useful in helping understand what data is available and how it is structured. The bottom inject queries a single room, change the room ID in the first change node.

[{"id":"3822a01a.859ea","type":"inject","z":"fb0c842.c87fc78","name":"","topic":"Wiser QUERY","payload":"","payloadType":"str",
"repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":620,"wires":[["2dfd4a60.601b06"]]},{"id":"fcb37ac1.cb36f8",
"type":"http request","z":"fb0c842.c87fc78","name":"","method":"use","ret":"obj","url":"","tls":"","x":770,"y":620,
"wires":[["c21166d6.4784c8"]]},{"id":"c21166d6.4784c8","type":"debug","z":"fb0c842.c87fc78","name":"","active":true,
"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":950,"y":620,"wires":[]},{"id":"a9652fa4.60f24",
"type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set","p":"method","pt":"msg","to":"GET","tot":"str"},
{"t":"set","p":"url","pt":"msg","to":"\"http://\" & $flowContext('wiserHost') & $.path","tot":"jsonata"},{"t":"set",
"p":"headers","pt":"msg","to":"{\t \"SECRET\": $flowContext('wiserSecret'),\t \"HOST\": $flowContext('wiserHost') & \":80\"\t}",
"tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":620,"wires":[["fcb37ac1.cb36f8"]]},
{"id":"2dfd4a60.601b06","type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set","p":"path","pt":"msg",
"to":"/data/domain/","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":620,
"wires":[["a9652fa4.60f24"]]},{"id":"9b02c89.7609c38","type":"inject","z":"fb0c842.c87fc78","name":"",
"topic":"Wiser QUERY Room","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,
"x":180,"y":680,"wires":[["38d8c719.5ae978"]]},{"id":"38d8c719.5ae978","type":"change","z":"fb0c842.c87fc78","name":"",
"rules":[{"t":"set","p":"path","pt":"msg","to":"/data/domain/Room/6","tot":"str"}],"action":"","property":"","from":"",
"to":"","reg":false,"x":370,"y":680,"wires":[["a9652fa4.60f24"]]}]

Get a simplified view of all of the rooms

This flow queries the ‘/data/domain/Room/’ path and then simplifies the output using JSONata

[{"id":"c8566325.8a07b","type":"inject","z":"fb0c842.c87fc78","name":"","topic":"Start Query","payload":"",
"payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":800,"wires":[["82a69e2d.bd5a1"]]},
{"id":"a3f3790.4883288","type":"http request","z":"fb0c842.c87fc78","name":"","method":"use","ret":"obj",
"url":"","tls":"","x":630,"y":800,"wires":[["2887038d.8bd21c"]]},{"id":"55b4a3e7.8eb21c","type":"debug",
"z":"fb0c842.c87fc78","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload",
"x":950,"y":800,"wires":[]},{"id":"a8da7133.b0ced","type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set",
"p":"method","pt":"msg","to":"GET","tot":"str"},{"t":"set","p":"url","pt":"msg","to":"\"http://\" & $flowContext('wiserHost') & $.path",
"tot":"jsonata"},{"t":"set","p":"headers","pt":"msg",
"to":"{\t \"SECRET\": $flowContext('wiserSecret'),\t \"HOST\": $flowContext('wiserHost') & \":80\"\t}",
"tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":800,"wires":[["a3f3790.4883288"]]},
{"id":"82a69e2d.bd5a1","type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set","p":"path","pt":"msg",
"to":"/data/domain/Room","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":289,"y":800,
"wires":[["a8da7133.b0ced"]]},{"id":"5d6dc6ef.f8cae8","type":"comment","z":"fb0c842.c87fc78",
"name":"Get basic room info for all rooms","info":"","x":190,"y":760,"wires":[]},{"id":"2887038d.8bd21c",
"type":"change","z":"fb0c842.c87fc78","name":"trim output","rules":[{"t":"set","p":"payload","pt":"msg",
"to":"payload{ \t $.Name : {\t \"id\": $.id,\t \"Name\": $.Name,\t \"CurrentTemperature\": $.CalculatedTemperature/10,\t \"DesiredTemperature\": $.CurrentSetPoint/10,\t \"Override\": $.OverrideType?\"Yes\":\"No\",\t \"OverrideTimeout\": $.OverrideType?$split($split($fromMillis($.OverrideTimeoutUnixTime),\"T\")[1],\".\")[0]:\"N/A\"\t }\t}\t",
"tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":790,"y":800,"wires":[["55b4a3e7.8eb21c"]]}]

The output looks like:

{
 "Bedroom 2": {
 "id": 5,
 "Name": "Bedroom 2",
 "CurrentTemperature": 19.2,
 "DesiredTemperature": 12.5,
 "Override": "No",
 "OverrideTimeout": "N/A"
 },
 "Master Bedroom": {
 "id": 6,
 "Name": "Master Bedroom",
 "CurrentTemperature": 19.2,
 "DesiredTemperature": 12.5,
 "Override": "Yes",
 "OverrideTimeout": "18:09:42"
 }
}

Control the system

These flows show you how to make changes to the controller from Node-RED.

Boost the temperature in a room for 30 minutes

Applies a 30 minute boost to 18.5°C for room id 1.

[{"id":"9d700a39.ebbaa8","type":"http request","z":"fb0c842.c87fc78","name":"","method":"use","ret":"obj","url":"",
"tls":"","x":770,"y":560,"wires":[["58845d6d.bf2a24"]]},{"id":"32c33bee.cf6c24","type":"inject","z":"fb0c842.c87fc78",
"name":"Wiser SET","topic":"","payload":"","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,
"x":140,"y":560,"wires":[["855deb47.8df6f8"]]},{"id":"58845d6d.bf2a24","type":"debug","z":"fb0c842.c87fc78",
"name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":930,"y":560,
"wires":[]},{"id":"6ec2a88e.839598","type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set","p":"method",
"pt":"msg","to":"PATCH","tot":"str"},{"t":"set","p":"url","pt":"msg",
"to":"\"http://\" & $flowContext('wiserHost') & $.path","tot":"jsonata"},{"t":"set","p":"headers","pt":"msg",
"to":"{\t \"SECRET\": $flowContext('wiserSecret'),\t \"HOST\": $flowContext('wiserHost') & \":80\"\t}",
"tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":560,"y":560,"wires":[["9d700a39.ebbaa8"]]},
{"id":"855deb47.8df6f8","type":"change","z":"fb0c842.c87fc78","name":"","rules":[{"t":"set","p":"payload","pt":"msg",
"to":"{\"RequestOverride\":{\"Type\":\"Manual\",\"Originator\":\"App\",\"DurationMinutes\":30,\"SetPoint\":185}}",
"tot":"json"},{"t":"set","p":"path","pt":"msg","to":"/data/domain/Room/1","tot":"str"}],"action":"","property":"",
"from":"","to":"","reg":false,"x":340,"y":560,"wires":[["6ec2a88e.839598"]]}]

The output from this contains updated room info.

You can easily amend the payload and path to any of the settings listed in the controlling the system section above.

Easily search for contributed nodes on the Node-RED Flows site

Julian Knight — Mon, 19 Mar 2018 22:00:00 +0000

Excellent work has been done to make sure that new nodes are discoverable. Here is a quick way to easily search from your browser.

The Flows website is the best place to go in order to find contributed nodes for Node-RED. It also contains example flows showing how to do various tasks.

You can search on that site by node name, author name and tags and the results are paged.

You can also use the URL to search and this means that you can add Flows as a custom search engine in your browser. Or even from Node-RED itself!

Chrome

Select “Settings” from the menu. Scroll down to the “Search engine” section and click on “Manage serach engines”.

Click on the Add button and fill in as follows:

How node.send() works

Julian Knight — Sun, 04 Mar 2018 22:56:00 +0000

Some detail not included in the documentation.

It is synchronous. See this discussion for details.

If you have code in a function node after a node.send(), it will not run until all downstream connected nodes have finished processing their sending routines. This is true of nodes further downstream as well.

Also see Issue 833 on GitHub

Another impact of this is that, because [msg’s may be passed by reference]({{ site.baseurl }}{% link nr-qa/msg-cloning.md %}) to downstream nodes, a subsequent node may alter the msg. This means that any processing after a node.send(msg) may be getting a msg object that has been altered.

Node-RED Q & A on Much Ado About IT

Node-RED communications protocols

The protocols

MQTT

Websocket

HTTP

TCP

UDP

UNIX Pipes

FAQ 4: How do I see the Node-RED log?

Running Node-RED as a “Service”

Linux

Windows

Running Node-RED manually

Using a “runner” to run Node-RED

Changing log output levels

Outputting your own information to the log

FAQ 3: What different ways are there to install Node-RED and which is best?

Global Install

Advantages

Disadvantages

Local Install

Advantages

Disadvantages

Containerised Install

Advantages

Disadvantages

FAQ 2: How can I run multiple instances of Node-RED on the same server device?

FAQ 1: Should I use a function node or a flow of core nodes?

A Glossary of Node-RED terms

Glossary

Creating a notification in the Admin UI

Options

Methods

Example

References

Cleaning up npm package install problems

Analysing the Performance of Node-RED

Platform

Back End (server)

Using Node.JS --inspect option

Front End Browser (Node-RED Admin UI)

Browser Performance Profiling

Front End (Dashboard, uibuilder, http-in/-out nodes, etc.)

A more comfortable way to edit the code for creating custom nodes

Installation

./bin/mergehtml.js

./node-src/node.html

How to create secure certificates

The problem

Possible fixes

It’s never simple

1. Get a domain

2a. Install a client for Let’s Encrypt

2b. Get the Cloudflare API details

2c. Get our first certificate

3. Certificate renewals

Using names instead of IP addresses

Using your certificate

Node-RED

settings.js

MQTT broker Mosquitto

/etc/mosquitto/conf.d/custom.conf

Webmin

Web servers

Router (Ubiquiti EdgeRouter)

InfluxDB and Telegraf

Grafana

How to secure Node-RED

Warning and disclaimer

Terminology

The basic architecture of Node-RED

Settings that you need to think about

Other considerations

NodeJS and package vulnerabilities

Issues and limitations

Specifics

References

Drayton Wiser heating control

Connecting to your controller

Using Node.JS `--inspect` option