Have we considered adding extra memory to the microcontroller that would allow additional locos to be run at the same time? I don't know what kind of memory that would be, but I imagine it may be possible. It would be really sweet to be able to add a shield to an Arduino and instantly have the capability to drive another 50+ cabs.
The arduinos are not designed to have memory added... But the code mods I am discussing mean that the code would use less memory generally and only need 4 bytes per cab instead of the 50 or so of the original version... So potentially hundreds of cabs ....
I think my best approach is to get to the point where there is a small, fast, reliable DCC API and then extend it for Railcom as required. This would be much easier than trying to include Railcom while trying to get the simple stuff working.
I'm currently working on porting to the SAMD21, which has 16x the amount of RAM as an UNO and 4x as much as the Mega... So it's not much of a concern. Can you please outline the changes you're making to the cab structure so we can understand how you're going to cut the memory usage so dramatically? I'm excited to see how many cabs I can get running on a SAMD21 with your modifications... Could be several hundred. Sent from my Pixel 3 using Tapatalk
Please let me know if the following looks correct.... I did some research on this and it appears that there are several interconnected factors. Keeping Railcom aside and sticking with basic DCC for the time being) 1) ACKs only happens on a prog track where the normal current is very low as its only driving one decoder and no engines so we really don't care about measuring the before-ack values. 2) The decoder performs a 60mA current pulse for a 1 response... 3) given the value obtained by analogRead of the shield... we need to know just 1 value that will detect a pulse... ACK_TRIGGER_PULSE_VALUE This is the smallest analogRead that will be considered a pulse and ACK_DEAD_ZONE_VALUE which is below the trigger value but above the resting current. 4) These values could be calculated from the shieldsmax rating and the cpu voltage or we could just measure a few cases and see what values we get. (Arduino shields put 3.3v on the analog pin at full shield current, but analogRead will return different values on a 3.3v processor or a 5v processor.) 5. We also need some kind of timeout value for how long we are prepared to wait for a pulse. I think the CV setting process should look like this: - send any recommended resets or idles. - send the programming message. - send any recommended resets... (the waveform generator will revert to sending idles as we are not feeding it with messages) Then.... something like this Code: long timeout=millis()+ACK_TIMEOUT; bool ack_found=false; while(timeout<millis() && ! ack_found) { delay(1); ack_found=analogRead(currentPin)>=ACK_MIN_PULSE_VALUE; } if (ack_found) while(analogRead(currentPin)>=ACK_DEAD_ZONE_VALUE) delay(1); return ack_found;
My version doesn't need packets with preambles and data bits at all, they are transmitted on the fly by the waveform generator. For the loco register, I have simply an array of {locoId, speed, direction} information (taking 4 bytes per loco) and the reminder messages are generated in the loop() process on the basis of one per loop as long as the waveform generator can accept a new message. Whenever a speed command is issued to the API, it adds/updates the loco in the loco register. Since this is fast and simple, there is no need for the caller to provide or know about register numbers so they are just ignored. This avoids the chaos caused if a loco is sent with two register numbers. Please be aware, I'm not, at this stage, modifying DCC-EX because I feel it's not mine to change as I don't know all the implications. But I'm trying to provide some technical solutions to the thorny problems of complexity, speed and memory. What I am doing is working from the basics backwards to create a completely new codebase structure that does the same job but sooooo much better. It's still open to others to either incorporate my suggestions into the EX code or cooperate to create a new system...
@UkBloke As I wrote in another thread, I just understand that everyone is free to make forks and improvements. I also believe that in the case of DCC ++, it may be appropriate to focus all useful efforts in one direction. At the moment the most common idea is to work together on the DCC EX project in favor of all those users, like me, who already own the DCC ++ system and who would be happy with an update that takes into account issues such as the speed of calculation, capacity increase (like Railcom) or compatibility with new cards and libraries available.
Sure, but don't build it such that you can't extend it in the future. This is not feasible for most MCUs unfortunately. That may depend on the decoders more than available memory. A sound decoder that hasn't received a packet within X milliseconds may start the power-down cycle. I missed commenting on the details above previously... In my CS code (not DCC++EX) I'm building the packet payload bytes only and deferring the conversion to DCC "on-the-wire" pulses to the transmitter code. This allows the packet transmitter code to be as efficient as possibly by using a pre-assembled preamble bit set and to process the payload bytes as-is without bit shifting etc. "b" and "c" sound like an over complication of the packet transmitter code and would shift responsibility of byte marker bits to loadPacket. "d" is a 100% violation of DCC spec, you can *NOT* count the preamble bit of the next packet as the "end of packet" marker bit. By dropping this bit you will need to send an extra preamble bit as the decoders will ignore a 10 bit preamble as an invalid packet if you intend on sending an 11 bit preamble (minimum required for OPS without RailCom). Similarly the 22 bit preamble for PROG would be truncated to 21 and then also be out of compliance with DCC spec.
By cutting out the complexity... one is left with something that is relatively easy to extend or modify. I think you will agree the current code is difficult! OK... I'll try and do the maths. Do you have any idea of the X milliseconds if no speed packet sent to a particular loco? I missed commenting on the details above previously... In my CS code (not DCC++EX) I'm building the packet payload bytes only and deferring the conversion to DCC "on-the-wire" pulses to the transmitter code. This allows the packet transmitter code to be as efficient as possibly by using a pre-assembled preamble bit set and to process the payload bytes as-is without bit shifting etc. ... compilcation? the data separator bits only took a 2 byte change to the code.. code and would shift responsibility of byte marker bits to loadPacket. Fair enough... I can send any suitable number of preamble bits, on each track.
100% the current code is very hard to follow and understand easily. I want to say it is in the range of 500 milliseconds but I could be wrong. It may not be consistent across decoders as it may be CV controlled. I found it via a bug in my transmitter code where it was stopping transmitting after X packets due to overload (since fixed).
@Atani As I recall you were looking for a DCC Api... To make it much easier to see, please take a look at https://github.com/Asbelos/DCCAPI where I hope you will find something to your liking. As before, its untested, my prototype did work but this code is in progress. I plan next to write a JMRI to DCCAPI routine to parse the string commands and call the API then it can be tested on my layout.
It was a deadlock in the code that was generating the signal which lead to all packets stopping. Another reason why the signal generation code needs to be as simple as possible (and partially why I rewrote that block of code a few times as part of progressively simplifying it). This looks interesting at a high level, I'd rather not give feedback on it in this thread but will send a few comments on the commits themselves.
I've actually got something very similar to @UkBloke that I just got working last night. Essentially turning the DCC commands into parameterized functions instead of passing a const char * and parsing it there. I think I've gone one step further to minimize the amount of memory required by specifying uint8_t and uint16_t where possible. https://github.com/davidcutting42/CommandStation Am I missing something or are @UkBloke and I doing the same work?
If you have a parser that can call a DCC api, then I suggest you take a look at using my API. If I get time this evening (AKA wife permitting) I'll take a peek at your parser.
Ok, let's get things back on track and work on our organization. We all have egos and we all have ideas and the hope is that we can put the egos aside, bring out talents to the table, and be aware of how we are coming across to others and the work they have done as we try to work together as a team. Our coding conventions, naming, feature changes, etc. all need to be agreed upon via a vote. As Billy Joel sings, "Don't go changin'..." the code without us discussing first our direction and how it fits with the plan. If you have an idea, suggest it, let's discuss it, and then once agreed upon, let's assign it. First, anyone wanting to contribute to the project, please read this. It is what we are about. If you can work in this framework, great, glad to have you. If you can't, understood hopefully we can still stay in touch: DCC++ Update Project 2020 Vision Statement, Scope, Mantra Keep in mind our vision and scope in particular when you are thinking of just what we can get a little Arduino to do and that we cannot be all things to all people. And then please read this: DCC++ Update Project 2020 Project Organization And lastly this: DCC++ 2020 Update Project documentation We can use help in all phases of DCC++ E; feature ideas, development, organization, web design and documentation. The Github repo has a website and more information and is where we are organizing most of what we need to move forward. The project task lists are here: https://github.com/orgs/DCC-EX/projects This is where we can add items (feature requests, bug fixes, etc.) and track their progress. If you are relatively new to our group or haven't previously volunteered, you can respond here with your goals and how you want to contribute or PM me and we will go from there. Let's create a topics of discussion list here so we have a plan for new features/projects for the next 3 months. For example, more work on current monitoring or enabling a Rocrail cutout, or actually reading Rocrail data, adding a new accessory command, etc.. Then we can add that to the project list when we are done shotgunning ideas.
To expand on the above. We have limited memory in the Uno. It doesn't mean we can't abandon it at some point, but we have a challenge if we don't want to fork it and have an Uno version and then all future versions are Mega (or better) from that point on. We are duplicating efforts and not coordinating with each other. Like replacing things with uint8_t, etc. I've also tried to factor code and found that sometimes that breaks things. I believe it to be timing related. But in particular, current monitoring is too fragile. How many locos do we need? Really? Some of you people are crazy I get clubs can have a lot of locos going, but how many people in their basement setup are running 50 trains? And if it is more than I think, is that our market? Again, can't be all things to all people. Where are the limits of the technology? DCC by definition can only transmit between 150-200 packets per second. It doesn't matter if you were running it on a supercomputer. In a best case scenario with no need to rebroadcast packets, etc at 50 locos that is 3 packets a second. Maybe not enough to account for missed packets and the wait until the loco gets its next packet. And what if people are running their accessories off the same track. So something to ponder. And while we are at it, one feature to at least discuss would be "packet prioritization"; assigning a value so that if packets are stacking up, we prioritize the type of packet or the command or type of decoder it is going to. If something needs more frequent updates, we make sure it gets them.
100% agree, on ESP32 CS I have a default limit of 30 "active" at one time. There is extra overhead beyond just the DCC bit stream to consider. I think 50 is an acceptable hard limit. With DCC++ and JMRI active you may have more problems though, JMRI at one point was forcing all "Active" locos to have function packets sent out on a very regular basis to "refresh" the decoders since DCC++ does not persist this data or transmit it except on change. On ESP32 CS v1.3.0 I added function refresh to the periodic updates and broke the packet queue code to the point that I scrapped v1.3.0 in favor of rewrite for v1.5.0.
You can save much more memory by changing sscanf(input,"some format string", values....) calls to sscanf_P(input, F("some format string"), values....... Anywhere you have a string constant its's important to see if it can be wrapped in an F() macro because that way the strings remain in flash memory rather than being loaded into RAM.
