Widgetlords Controller - Node-Red, Raspberry Pi Controller Build

[FischAutoTechGarten]

Can you expand on the name changing with Termite? I wasn't aware the burned in data could be changed.

Maddy,
You can't change the 'Device' that you receive when you query the EZO circuit using i <cr> command. Think of the burned in 'Device' parameter as more of a Device Type or Object Type. So every RTD EZO circuit will have a Device or Type of 'RTD'.

However, you can change the 'Name' parameter. Think of the 'Name' as the Device Instance or Object Instance. So you could have two RTD Types... both would have a 'Device' of RTD, but one could have a 'Name' of TankTemp and the other could have a 'Name' of StorageTemp, to cite an example.

The command for name is Name,zzt <cr> if you query the EZO circuit using Name,? <cr> you will receive ?Name,zzt. So in my example above I'd use Name,TankTemp <cr> and Name,StorageTemp <cr>.... then when Scanning I would issue the Name,? <cr> command for each IP address encountered (in the same manner that you are fetching Device with the i <cr> command).

I use Termite (a lightweight RS232 terminal, www.compuphase.com) and one of the Atlas-Sci USB EZO Carrier boards to do the initial setup. You start with the EZO circuits in UART mode and place them in the Carrier board and plug a USB cable from your laptop/PC into the Carrier board... (It's an ODD style of USB header so you'll need to go through your kit to see if you've go it). Generally, Windows will assign COM3 to the virtual serial port and just have Termite connect to that COM3 using basic 9600,8,1, none,none settings. It's also during this setup where I would change the I2C address for the 2nd Instance of a Type, rather than have duplicates of the that type's default address on the Bus. So perhaps TankTemp keeps the 102 Default, but StorageTemp is assigned a new I2C of 92. It's the Dosing Pumps where this scenario is most likely.... as most will do a minimum of Macro/Micro nutrient dosing..

The DataSheets only show the Name command available for UART connections. However, the commands work under I2C as well. I verified with Jordan and Ephraim that they were going to keep that working as they recognized a scenario where scanning the bus and dynamically obtaining Types and Instances was a good thing..

 

[theatrus]

Continue to be impressed at the setup - Node-Red is a modern day LabView and its less insane to boot

 

[Nanorock1970]

Thoughts on analog inputs?

I like this simple dashboard but from an automation engineer most of my clients want a graph to see history...I do like this though, maybe a click through to a history chart of the analog inputs to keep this a simple display for instant feedback and a more detailed page for the graphs....??

 

[Nanorock1970]

This is great stuff.....Looks like I will be PI driven either with widgetlords or reef-pi on my next tank build..Watching for updates here....

 

[MaddyP]

I like this simple dashboard but from an automation engineer most of my clients want a graph to see history...I do like this though, maybe a click through to a history chart of the analog inputs to keep this a simple display for instant feedback and a more detailed page for the graphs....??

How often do you log data from an input which changes nearly every few seconds? I would think the dataset would become unmanageable?

 

[Ranjib]

Excited to see the progress , keep it going.

As for reef-pi integration I am not sure about widgetlord integration, since we support ezo circuit, and relays are just GPIO, i think it might work as it is. We try to keep reef-pi humble, opensource, affordable but effective and approachable solution. These core values is reflect by our tech choices. Raspberry Pi, go, react are the flagship projects in each of their domains. Each of them are not the best in terms of sophistication (e.g. C is more performant than go, and nodejs easier than go), instead they hit the middle ground where longevity and price point/maintenance gives best return of investment With this capital (the software, rudimentary electronics and the awesome community) I prefer to tackle the user facing challenges more than sophisticated techs. We'll take on the hard tech challenge if the solution demands, but not simply because its possible or its little bit better in some aspect. PLC style industrial designs are more durable, but those are overkill for home usage, i would use if i have them laying around, but not for new things. They are prohibitively expensive. This makes me less excited about widgetlord type integration other.

For example, the reef-pi ph board development was most delightful thing I feel. It not only gave us an opensource design of ph board at half the price (compared to ezo) but also a starting point for ORP and other sensors. After using it for two years, I'm not sure if atlas is any better than the opensource sensor, hence i dont find it very exciting to tinker with atlas scientific products any more. I want to see lot more improvements in reef-pi, but mostly around the challenges we face as reef keeper(for example the temperature sensor is finicky , so we might want to support some alternatives etc) or new ).

 

[Nanorock1970]

How often do you log data from an input which changes nearly every few seconds? I would think the dataset would become unmanageable?

Data history can be done in two ways for the system I work on. Either change of value or incremental time based collection. Even the change of value can be given a range such as +/- .5 change or +/- 1 change....etc. Otherwise, yes you will have a pyle of data to display. Also we use rolling records such as say 500 values then roll the oldest or archive the data history at some period such as once a week. The second option is for customers that need to look back at values longer then are stored in a rolling value option.

 

[theatrus]

Data history can be done in two ways for the system I work on. Either change of value or incremental time based collection. Even the change of value can be given a range such as +/- .5 change or +/- 1 change....etc. Otherwise, yes you will have a pyle of data to display. Also we use rolling records such as say 500 values then roll the oldest or archive the data history at some period such as once a week. The second option is for customers that need to look back at values longer then are stored in a rolling value option.

I deal with time series databases (write them, work on metrics plumbing, etc) as part of the day job, so this strikes home.

Sliding buffers are great for high resolution data - the reality is you don't really care about exact second by second plays for a sensor or input forever, so providing a fixed space prevents runaway growth.

What is important over time is then down-sampling the data. Say you had a high resolution circular/sliding buffer of events, you can compact them into a few values as an aggregate instead. For an on-off sensor or an analog sensor, taking the average and min/max over chunks of time is super common. For a binary sensor, the average even tells you the ratio of on time to off time. You can roll these chunks up at different time blocks (minute, hour, day, etc) and they cost effectively nothing to store. A very common old-school but still functional appraoch to storage is Whisper files ( https://github.com/graphite-project/whisper ), an implementation of a "round robin database".

No one should use whisper files for large tech monitoring, but for sensor data storage for 10-50 time series its almost perfect.

 

[FischAutoTechGarten]

Maddy, I see you haven't posted in a bit, but I thought I would share a small experience I had.

I'm using a screen toggle in NodeRED to turn on/off my UV Filter and Pump... as I don't have any specific automation yet (no feed mode / water change interlocks). I was performing an Inject of True into the Toggle Object, so that the intial command was ON when the RaspberryPi/NodeRed start up. Because I'm using a relay board over i2c.. it wasn't always ready to accept it... and I'd have a startup event where the toggle switch was in the ON (true) position on the screen, but the pump was not running.... (a 'status lamp' on the screen reported that the relay was not engaged). Yes, I have a .56uF capacitor for induction suppression on the pump (Co2 valve too) so that spikes don't interrupt comms... (and it also helps relays last longer).

I have 8 relays on an NCD i2c expansion board. I don't have this issue with other devices on the relay board (heaters, co2 valve), because there is logic that runs continually that makes decisions and this seemed to work well.

Here's what I did:
At startup, I wait 5 seconds before scanning any sensors.
In that 5 seconds: I wait 1 second and command the Pump Off via an inject into the onscreen toggle switch (obviously the pump is off because we are coming from a restart). I wait one second and command the UV Filter Off in the same manner. I wait 1 second and I command the Pump On, and wait another second and command the UV Filter On. 1 second later.... the sensor scanning starts.

This works.. Within 5 seconds the entire system is at it's startup defaults and measurement/control is happening. I know you are doing remote comms.. and relays... so I wanted to share one of my tribulations with you.

 

[MaddyP]

Maddy, I see you haven't posted in a bit, but I thought I would share a small experience I had.

I'm using a screen toggle in NodeRED to turn on/off my UV Filter and Pump... as I don't have any specific automation yet (no feed mode / water change interlocks). I was performing an Inject of True into the Toggle Object, so that the intial command was ON when the RaspberryPi/NodeRed start up. Because I'm using a relay board over i2c.. it wasn't always ready to accept it... and I'd have a startup event where the toggle switch was in the ON (true) position on the screen, but the pump was not running.... (a 'status lamp' on the screen reported that the relay was not engaged). Yes, I have a .56uF capacitor for induction suppression on the pump (Co2 valve too) so that spikes don't interrupt comms... (and it also helps relays last longer).

I have 8 relays on an NCD i2c expansion board. I don't have this issue with other devices on the relay board (heaters, co2 valve), because there is logic that runs continually that makes decisions and this seemed to work well.

Here's what I did:
At startup, I wait 5 seconds before scanning any sensors.
In that 5 seconds: I wait 1 second and command the Pump Off via an inject into the onscreen toggle switch (obviously the pump is off because we are coming from a restart). I wait one second and command the UV Filter Off in the same manner. I wait 1 second and I command the Pump On, and wait another second and command the UV Filter On. 1 second later.... the sensor scanning starts.

This works.. Within 5 seconds the entire system is at it's startup defaults and measurement/control is happening. I know you are doing remote comms.. and relays... so I wanted to share one of my tribulations with you.

It has been a while! Over Christmas my wife and I moved to the PNW so setting up the tank has been the priority. Thanks for the heads up! I'll keep this in mind when I think through restarts, though I have my system polling every 3 seconds to confirm everything is in the correct state.

 

[MaddyP]

Now that the tank is properly setup, I'm back on developing Node-Red! Thoughts on the "refined" UI components?

The outlet/switch is a fork of "node-red-contrib-ui-multistate-switch" with improvements to show label / input state above, customizable colors, options, and labels. And the graph was updated to utilize Highcharts instead of Charts.js.

 

[MaddyP]

Made some progress on the "outlet" over the weekend! Decided to go full custom by merging 'node-red-contrib-functionx' into the new "outlet" node. This would allow the user to define which option would run through the function and how frequently the function should send a new value.

Still working on functionality, I will publish this node once it is working as part of a portfolio of reef widgets (outlet, switch, probe...).

 

[FischAutoTechGarten]

Awesome work! I would imagine the authors of the multi-state switch would be willing to make your modifications part of their project. They are very open to suggestions, I mean they started the whole effort becuase I asked if a Node existed that looked like the toggle switch that MaddyP built! (I go by my business name of SonoraTechnical on the NodeRed forums) LOL. One of the authors is on a break for about another month so you may not get much input for just a bit from him. Either way, looks like I'll be trying out your node.

I'm finally enabling my Atlas-Sci EZO-PMPs in NodeRED. I'll publish some code when I get that working. I cheated and did all of the configuration and calibration using Termite over UART.... instead of building a dashboard to accomplish it. Now I need to finish the flows and dashboard for them as I have them attatched to the PiandNodeREd via i2c. I have 3 of them, so I'm going to handle the device scanning and 'probe' assignments a bit differently.. I have to rely on NAME, instead of Device that is returned by STATUS in order to differentiate the multiple pump device instances.

Note: I'm probably going to draft very simple flows/dashboards for all of the EZO devices and submit them to Atlas-Sci and WhiteBoxLabs so that they can make them available for download. I'm going to credit you becuase you wrote the calibration portion that I use in my own flows. So send me some private PMs about how you would like me to mention you... Conversely, if you've already begun such an effort, let me know.. and I'll augment it.

 

[MaddyP]

Awesome work! I would imagine the authors of the multi-state switch would be willing to make your modifications part of their project. They are very open to suggestions, I mean they started the whole effort becuase I asked if a Node existed that looked like the toggle switch that MaddyP built! (I go by my business name of SonoraTechnical on the NodeRed forums) LOL. One of the authors is on a break for about another month so you may not get much input for just a bit from him. Either way, looks like I'll be trying out your node.

I'm finally enabling my Atlas-Sci EZO-PMPs in NodeRED. I'll publish some code when I get that working. I cheated and did all of the configuration and calibration using Termite over UART.... instead of building a dashboard to accomplish it. Now I need to finish the flows and dashboard for them as I have them attatched to the PiandNodeREd via i2c. I have 3 of them, so I'm going to handle the device scanning and 'probe' assignments a bit differently.. I have to rely on NAME, instead of Device that is returned by STATUS in order to differentiate the multiple pump device instances.

Note: I'm probably going to draft very simple flows/dashboards for all of the EZO devices and submit them to Atlas-Sci and WhiteBoxLabs so that they can make them available for download. I'm going to credit you becuase you wrote the calibration portion that I use in my own flows. So send me some private PMs about how you would like me to mention you... Conversely, if you've already begun such an effort, let me know.. and I'll augment it.

Thanks! Although these changes do add functionality to the multistate switch, that project fits the needs of most users looking for a multistate switch in a simple way. It might be best to publish the changes as a separate switch since it is meant to fill a very specific purpose. Over the next few weeks I plan to create a whole portfolio of "reef" widgets including outputs, inputs, and probes so a user can build a complete dashboard. These different widgets will be able to reference one another by name in the function area.

Looking good with the EZO peristaltic pumps! The one I have for testing is really loud...are yours noticeably noisy as well? Wish Ezo offered a better peristalitc pump, maybe even a stepper. I've ordered two of the GHL Doser 2.1 heads/motors to integrate into my project, these should be nice and quiet.

Great idea with publishing the flows for Ezo/Whitebox! I am thinking about creating some custom nodes for these as well so we don't have to rely on a complex flow. But they will have to wait until the 'reef' widget portfolio is done. To many projects, so little time! Feel free to reference 'MaddyP' and if I can offer any support please reach out!

 

[MaddyP]

Awesome work! I would imagine the authors of the multi-state switch would be willing to make your modifications part of their project. They are very open to suggestions, I mean they started the whole effort becuase I asked if a Node existed that looked like the toggle switch that MaddyP built! (I go by my business name of SonoraTechnical on the NodeRed forums) LOL. One of the authors is on a break for about another month so you may not get much input for just a bit from him. Either way, looks like I'll be trying out your node.

I'm finally enabling my Atlas-Sci EZO-PMPs in NodeRED. I'll publish some code when I get that working. I cheated and did all of the configuration and calibration using Termite over UART.... instead of building a dashboard to accomplish it. Now I need to finish the flows and dashboard for them as I have them attatched to the PiandNodeREd via i2c. I have 3 of them, so I'm going to handle the device scanning and 'probe' assignments a bit differently.. I have to rely on NAME, instead of Device that is returned by STATUS in order to differentiate the multiple pump device instances.

Note: I'm probably going to draft very simple flows/dashboards for all of the EZO devices and submit them to Atlas-Sci and WhiteBoxLabs so that they can make them available for download. I'm going to credit you becuase you wrote the calibration portion that I use in my own flows. So send me some private PMs about how you would like me to mention you... Conversely, if you've already begun such an effort, let me know.. and I'll augment it.

You should start a build thread so we can see your setup in more depth!

 

[MaddyP]

Making real progress with the UI nodes, though I had to update the Pi to ARM64 so all 8G's of ram could be used. Node-red-dashboard is hungry! But there was an issue...Raspi has been testing ARM64 OS for some time now and it runs well, however Widgetlords does not have a ARM64 build for their library nor are they planning to release one until the RaspiOS ARM64 is stable. Since libwidgetlords is critical for this project, I had to take the time to learn how to build from source just to create a ARM64 install:

GitHub - MaddyTP/libwidgetlords_arm64

Contribute to MaddyTP/libwidgetlords_arm64 development by creating an account on GitHub.

github.com

Now back to the UI. Outlet is functional with ability to load external Node modules! Still some refining to complete before publishing but well on the way!

 

[MaddyP]

It's starting to look real nice!

 

[MaddyP]

Getting close to having the UI nodes complete, still some work to be done on the input/output nodes to account for analog values. Probe node accepts single value/array and displays the most recent value as a text readout, limited by the timeframe setup in config.

 

[FischAutoTechGarten]

You should start a build thread so we can see your setup in more depth!

I have a build thread for the FishNook on plantedtank.net (same userid).... But I need to do an indepth one on my aquarium controller.

I submitted some super basic flows to WhiteBox/AtlasSci where all of the commands and addresses are hardcoded.... Just so they could see what it took to read/write to them. I know they are not as excited by NodeRED as myself simply becuase it's difficult to secure the devices and protect your IP. But they've had a number of customers using NodeRED and they needed basic flows...

Doing a bit of refactoring on my flows.... I adopted your approach to scanning for IPaddresses and then reading in the Device Type using the i cmnd (for info) and extracting the Device Type. This works fine, until you introduce a 2nd RTD and two more PMPs like I have done. So as part of the refactoring, I'm using the Name cmnd to extract the names. Caveat is of course you have to setup your devices before hand with a standard naming convention that you adhere to. (I'm simply doing PMP1, PMP2, PMP3, RTD1, RTD2, pH1, ORP1, etc...). I also have some ideas with doing a 'pseudo instance enabling' where I can handle upwards of 4 of each device type. Just got to figure it out.

 

[FischAutoTechGarten]

Looking good with the EZO peristaltic pumps! The one I have for testing is really loud...are yours noticeably noisy as well? Wish Ezo offered a better peristalitc pump, maybe even a stepper.

Yes, they are a bit noisy.... They have released a new, larger pump called, PMP-L . I haven't tried it as I'm fine with the three that I have.

I've been testing out a new MarkII Tentacle-T3... White Box Labs sent it to me to evaluate for them. It supports the EZO-FLO stamp with a small 3 screw terminal for attaching flow meters. They also have a nice 'aux' Connector that allows you to extend the i2c bus beyond the board. Looks good. This is a prototype, so it will be interesting to see how it looks in it's production format. They've got a few more underdevelopment... They showed me schematics of a new one for the Pi-ZERO-W... I like that one... They also spilled word of a Tentacle-T8 (but didn't have anything to show me)..

Atlas-Sci is also selling a little 'bus expander of sorts'.. or 'i2c hub of sorts' called the Sensor bridge... Really useful for the 5 pin devices (like EZO-PMPs, EZO-HUM) devices.... as you just make one home run back to the RaspberryPi or Tentacle board.

So yeah, I'm definitely bought into the atlas-sci and ncd-io 'ecosystem'.. LOL.

Surprised you felt like you had to go all the way to 8GB.. My flows are super complex (albiet not as much trending as you are doing), and I'm finding my 4B w/ 4GB to be sufficient for now. I'm going to work with InfluxDB, Grafana and MQTT on this Raspberry Pi in conjunction with NodeRED.... but alas, I need to finish up this refactoring I'm doing now.