[Back to Home Page]
www.RomanBlack.com
"BlackNet" serial network protocol
A system for unlimited number of devices to transmit/receive serial
data on 1 wire.
Originally written May 2009, added more diagrams and finally put on the net 27 Apr 2011.
What is it?
This is a system for many devices to be attached to one serial bus wire,
where any devices can send/receive at will with no conflicts. I designed
this for use with my home automation controllers, but it has other uses
like multiple sensors or multiple controlled devices.
All information on this "BlackNet" page can be considered open source and used
for whatever, but please mention me if reproducing any of this work.
Hardware
The hardware side is very simple. One common ground. One "bus data" wire which
is pulled up to 12v (or 13.8v) by a pull-up resistor. I ended up using a
constant current pull-up of 150mA, so that I can use devices powered from
the bus wire itself. This means I only need a 2-wire cable for long runs
like temperature sensors on my solar panels etc, as they only draw a
couple mA each they can use "ghost power" from the bus which saves running
power cables to these sensors. Most of my devices have 12v power cabling
too so the ghost power is just an option.
The BlackNet bus data line is both a transmit and receive line, for all devices.
Transmit. Each device can transmit by pulling the bus wire low
to send one or more serial bytes. I used standard 8N1 serial, 1 start
bit, 8 data bits and 1 stop bit. If no device is transmitting, the bus wire
remains high at 12v.
Receive. Each device can receive any byte which is on the bus.
So any device is capable of hearing what is said by any other device.
Desired protocol system
What I wanted from this system;
Only 2 wires needed; bus and ground
No limit to number of devices*
Easy serial using UART or PIC bitbanging
No special hardware needed, just a PIC and a transistor
No limit on transmission distance or baudrate*
Ghost power option
Any device can listen if it chooses
Any device can transmit, or choose NOT to transmit
Transmitting device can transmit any number of bytes
If a device fails it doesn't affect other devices*
Devices can be added or disconnected at any time with no issues
Multiple masters (transmitters) with no issues
Multiple slaves (receivers) with no issues
Devices do not need serial receive capability or UART to work on the bus
All devices transmit in sequence
A failsafe way of identifying all transmitting devices
If any devices transmit garbage bytes it does not affect other devices
I should clarify some points;
There are no limits in the protocol to number of devices
although some limitations will occur because of line loading.
It would still be reasonable to have hundreds of devices on the
network if it has been set up for that.
The protocol itself has no limit on distance. For long distance networks
the devices can just use a lower baudrate.
A device that fails will not affect the network provided that it
does not short circuit. Any other device failure including; loss of power or
open circuit or fail to transmit or garbage transmit will have no
effect on other devices.
No device is REQUIRED to transmit.
This is important, it means any device can transit only if it chooses
to, without affecting the chain of other devices sending/receiving.
This is good for devices sending an infrequent alarm or sensors that
only need to "report in" after long intervals. It also means that if
any device fails or is disconnected there is no effect on the other
devices. Obviously this is also good for devices that are in
low-power mode most of the time.
All devices CAN listen, but no device is REQUIRED to listen.
This is a big strength of the protocol. It means that a device
that sends data on the bus is not required to have a UART to
receive bytes from other devices. All it needs is to monitor that
the bus is free for X amount of time.
Any device can be a transmitter or a receiver or combination of both.
This is designed so that some devices might be light switches
(transmit), some might be lights (receive) some might do both tasks.
There can be many controllers on the bus controlling the one
receiving slave or vice versa. The system is designed to be very open-ended.
The protocol
I won't claim any great originality here, over the years I have seen many
systems for a number of devices on one serial wire so I just copied different
elements from existing systems to combine together to create a very open ended
and simple 1-wire network to suit my needs.
It is basic and sequential ie a "round robin"; No device can transmit until device1 transmits.
Then device2 can transmit IF it chooses, as many bytes as it chooses.
Then device 3 can transmit IF it chooses, etc.
Every device gets a chance to talk, then that "loop" is over.
For my home automation I chose to make device1 the master clock, so every
1/2 second it transmits the full date/time etc. So devices can only transmit
once per 1/2 second. The protocol does not require any specific loop timing,
it is only required that every device has had its CHANCE to talk,
then device1 talks again to start the loop all over again.
Making sure all devices talk in turn!
Each device knows its own number. After device1 finishes talking, the bus
goes free again.
After the bus has been free for a period of T*2 then device 2 is allowed
to start talking (if it chooses). Then after the bus has been free for
a time of T*3 then device3 is allowed to start talking. Then after the bus
is free for a time of T*4 device4 is allowed to talk and so on.
If any device chooses NOT to talk, then one time period T later the next device
can talk so there is no penalty in bus time. This is very efficient for devices
like light switches etc that only need to talk when someone presses the
switch.
Yes this can become slow with a very large number of devices IF they
all need to talk at once, as the total bus loop time is an exponential
function of number of talking devices;
The total number of wait periods (T) for any number of devices (n)
is roughly given by the formula;
total_T = n * (n / 2)
So for 10 devices the total T time needed will be 50 T.
For 100 devices the total T time needed will be 5000 T.
(However this is only true if ALL 100 devices talk in that cycle.
In a best case cycle where none of those 100 devices talk the total
cycle time is only 100 T.)
As the time period T is only a few uS this is still fine for 100+
devices to all talk
in a 0.5 second bus loop time. The protocol is based on minimal hardware
and open-endedness, speed comes last. The open-ended nature of the
protocol allows any or all devices to NOT talk so there is an option for
very large numbers of devices on the bus if they don't all need to talk
in every loop. Obviously, there is no time used up by devices
that only listen!
About time period T and when a device knows it's turn to talk
Originally I was going to use a large time period for T that is longer
than the total time for one serial byte, so that some high bits in a byte
are not mistaken for a finished transmission.
However it only requires ONE safe period this length after each transmission
to ensure transmission has finished, then after that the period T can be
shorter than a serial byte. A period T of 20uS is fine even with cheap PICs
running from 4MHz xtal (1uS timing). The single safe period should be longer
than a serial byte, for example can be 200uS if using 56kbaud serial.
Procedure for a cheap transmitting device detecting time of 7T;
1. first, wait for any device talking (bus low)
2. wait for bus going low-high, then reset timer
3. if bus goes low again go back to 2
4. if timer reaches safeperiod + 7T, start transmission!
Practical uses
Obviously this is not a good system for devices that need to send lots of dense
data. However it would be a good system where the devices need to be cheap
as the devices do not need UART to be able to talk on the network. They only
need to sense the TIME the bus is free, then transmit a data packet using
bit-banged serial.
This could be useful for a very large number of cheap sensors on a 2 wire
bus, like solar panel sensors that only need to talk occasionally to signal
that each panel is working, or signal a panel fail condition.
It can be ideal to network cheap sensors within a building, like hundreds
of light switches (or people sensors) where they only need to talk if a switch
or sensor is activated. The same network can have thousands of receiving
devices (like smart lights) that can just listen to the bus to see if they are
commanded on or off.
It might be useful for industrial or alarm sensors, especially as all
devices are in parallel there is no problem attaching or detaching devices
to/from the bus at any time while the bus is running.
Practical implementation for my own network
I decided to make device1 be a system clock. This transmits a time code
of "year-month-day-HH-MM-SS" every 1/2 a second, and always starts a bus
cycle every 1/2 a second.
All devices afer device1 can know the time (if they receive bus data)
and this allows for obvious time related features where devices can switch
on/off at specific times without those devices needing to talk on the bus.
Another benefit is that devices can talk only on specific cycles, so a
device that only needs to talk every 30 seconds (like a room temperature
sensor) can do so and uses very little bus time in total.
Another simple trick I added later was to make every device transmit its
device number as the first byte (assuming it is a talking device). This is not
required by the protocol but seemed useful for a number of reasons.
For hardware I used a constant current bus pullup of 150mA, to 12v.
The 12v is unregulated battery voltage generally 11v to 13.8v range.
I have not used ghost power in any devices so far, as I used telephone
style cable and run these 4 wires;
+12v unreg
+9v reg
Data bus (pullup +12v 150mA)
Ground
As the bus data is normally high (same polarity as a PIC USART),
it can be connected directly into a
PIC USART RX pin through a 47k resistor. This allows minimal hardware
receive with only one resistor (uses the PIC internal port pin diode).
Output to the bus is through a simple NPN transistor that is capable
of pulling the 150mA bus line low. That means pretty much any cheap
small-signal NPN like a BC337. If the serial output is bit-banged
this only requires one resistor and transistor, and the bit banged
serial from the PIC output pin must be inverted compared to a
normal PIC USART TX pin.
However if the output
is from the PIC USART TX pin a second transistor is needed for data
inversion. I figured that for minimal devices they can bit bang the
output, and more complex devices can afford another 5 cent transistor.
The benefit is that the RX does not need inversion, and the bus line is
normally high allowing ghost power.
Problems and limitations
Obviously this network is for small data packets, a handful of bytes per
device. Although any talking device can transmit any number of bytes
generally they should be transmitting a small amount of data.
As it is a "round robin" system (taking turns), devices need to know
their own device number and have individual device numbers. This was fine
for my needs and should be ok for permanent installations but could be a
big limitation in some systems. It should be possible for a master to
track all device numbers in use, and identify new devices and assign
them a number although I didn't bother with auto numbering and just
manually assigned numbers to each node I built.
There is a problem if a device fails short circuit as they are parallel
on the bus the entire bus goes down. However this type of fault applies
to most types of network busses and is easy to trace. A device that
fails open circuit will not affect the bus.
There is a time problem if there are a lot of devices that must talk
during one cycle. My home system with 0.5 seconds per cycle is limited to about
100 devices talking at once. However most devices do not need to talk EVERY
cycle, and bus time can easily be distributed this way. There are plenty
of applications that can have a large number of talking or listening
devices without all the devices needing to talk at once.
A big speed improvement
Because I made the talking devices transmit their number, this allows
a "fast mode" where device8 does not have to wait the typical 8T time
before talking. If it listens and hears device7 has just finished talking
then device8 can talk immediately (as it knows it is next).
Like most of the protocol this "fast mode" is open ended and any device
may use fast mode or not without the entire network needing to be
structured this way. It would be most useful for higher numbered devices
that might be clumped in groups (to talk in groups). Of course if there
are only 20 or 30 must-talk devices on a network then fast mode is really
not needed.
Ghost power
Because the bus line spends most of its time high (especially when there
is a lot of devices!) it is ideal to provide ghost power to small
low-power devices.
The bus would charge caps in all devices and provide enough power to
run the devices. With low power PICs and low quiescent current regulators
common and cheap now it is conceivable for a simple 2-wire bus to power
a large number of devices as well as carrying data.
A practical ghost power circuit in a device would have a diode and resistor charging
a large cap to approx the bus high voltage (12v). The resistor is needed
to reduce peak currents after any 0 bit. As the data 0 and 1 bits are detected
at logic levels (approx 1v and 3.5v) the data on the bus will not be affected
by the ghost power charging caps etc which will ideally occur in the
9v to 12v range. The cap in each device should be large enough so that
it's voltage does not drop much below 9v when the bus is transmitting
data.
Ideally a 5v (or 3.3v) low quiescent curent low dropout voltage
regulator would be used. If so, a device might use less than 1mA average
current from the bus so it would be possible to have a quite a lot of
ghost powered devices on a network. Although I like the idea of ghost power
I did not use it on my internal 4-wire home network. I may explore ghost power
devices with an external 2-wire network at a later date.
My home network
I won't go into the exact details of my network setup as it performs some security
tasks but here is a glimpse at the PCB I whipped up in 2009 for my BlackNet
master clock (device1).
At the right you can see the 4 wires to the network system, and in the
circled area you can see that the serial output system just uses 2
SMD transistors and a few resistors.
The master clock also has a multi-digit display and a Dallas Semi 32kHz
temperature compensated precision oscillator module to give extremely
accurate timekeeping. It is accurate to 2PPM which is around 1 minute
a year error! I also have a GPS module that I will eventually
connect as another node on the network that will act as a second "clock"
and send error data to the master clock and allow the master clock to
gently adjust it's time to re-synchronise to match the GPS clock.
Minimum parts count - multiple devices
(minor update 30th Apr 2011 - in response to a question I was asked about minimal hardware applications).
For an extremely low parts count solution you can connect multiple
microcontrollers on the one data bus line and use a 5v data bus. This
uses a simple 270 ohm pull-up resistor for the entire data bus.
This method can be used for multiple PICs on a single PCB or within a single
device (like a large display unit).
All that is needed is one diode per PIC microcontroller if you are using
the USART TX output. If you are bit-banging serial output on low
cost PICs (like a multiple sensor array) then the diode is not needed
as the firmware can toggle the serial output between pull-down and
high impedance states.
This simple hardware will still allow hundreds of devices on the one data bus
line, although the 5v line is not suitable for ghost power and will not
generally allow the long distances obtainable from a 12v line and constant
current pull-up.
- end -
[Back to Home Page]