Firmware architecture for firmware update

Hello.

I am implementing a program to be updated via GPRS channel. The total siz
of the firmware reaches about 256 kB which makes it somewhat costl
upgrades. I considered the idea of dividing the firmware into independen
parts that can be updated independently and upgrade only the changed parts
but I don't know how to begin. Does anyone has had a similar problem? Ho
do you have solved it?

Thanks.



---------------------------------------
Posted through http://www.EmbeddedRelated.com

On 05/20/2011 04:17 AM, alacky3 wrote:
> Hello.
>
> I am implementing a program to be updated via GPRS channel. The total size
> of the firmware reaches about 256 kB which makes it somewhat costly
> upgrades. I considered the idea of dividing the firmware into independent
> parts that can be updated independently and upgrade only the changed parts,
> but I don't know how to begin. Does anyone has had a similar problem? How
> do you have solved it?

Under windows you'd do this with dll's. Even though I use Linux, I
can't remember the name of the same concept, but the file suffix is .so.
At any rate, whether your OS supports it or not you'd need to have
nice logical modules well separated by function (so that upgrading one
thing doesn't break all the rest) and a means of dynamically linking the
separate bits.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

256KB on GPRS is not excessive. It will finish in less than 4 minutes
worst case (2G single timeslot).

If you really need faster updates or less bandwidth usage, you could
inspire with RSYNC. It improves throughput for files of which a
"similar" version is already present on the other side of the link.
What definition of "similar" works best for your application, depends
on the details of your project.

You can even improve a lot over RSYNC, if you know exactly what's on
the other side (by firmware version identifier). You could then
implement a "patch processor" (interpreter with defined command set),
and produce a patch instruction stream, which you then optimize,
compress and transmit. On the other side of the link, it is
decompressed and executed, and patches the existing firmware into the
new version. Done right, this is the quickest/shortest possible way
of transmitting the update (because it is a compression algorithm
optimized to the specific application at hand).

However, I suspect that standard compression algorithms on the binary
firmware image and possibly a recommendation of using better wireless
technology (EDGE or 3G) is quicker to deploy than either of these, and
the user will not notice much of a difference anyway.

On 5/20/2011 4:17 AM, alacky3 wrote:

> I am implementing a program to be updated via GPRS channel. The total size
> of the firmware reaches about 256 kB which makes it somewhat costly
> upgrades. I considered the idea of dividing the firmware into independent
> parts that can be updated independently and upgrade only the changed parts,
> but I don't know how to begin. Does anyone has had a similar problem? How
> do you have solved it?

Presumably, you don't have a "spare" 256KB in the device to
cache the image(s) -- persistently -- until *all* of them
can be downloaded?

If you try to do this piecemeal, you have potentially several
hazards to avoid -- all having to do with incomplete or partially
complete images being available.

First, you have to have AT LEAST enough "cache" (though *this* can be
volatile) to store the largest "piece" that you will be downloading.
This safeguards against the case wherein the image can't be downloaded
intact in a single piece (i.e., you can't risk "updating *while*
downloading" for fear that the image is corrupted or interrupted
*during* the download -- you need the previous "working" image
to be available for your device to continue functioning in the
event that the image isn't downloaded completely/correctly).

You also need to have some strict sense of configuration management
that allows you (your device and/or the device serving up these
images) to know what the device in question's *current* configuration
is and which "pieces" can be applied to that configuration to yield
"operational" (albeit possibly with other "known bugs") configurations.

For example, if the user is at version A.0, possible upgrade
paths may be A.0->A.1->A.3->A.7->B.0, or A.0->A.2->A.7->B.0
depending on which "modules" are updated along the way. Note
that the path taken will determine which functionality (and
"bug-onality") the device retains *while* it transitions to the
final state (recall that the process may take minutes, hours,
weeks, months or YEARS! Since you are allowing it to be
"interrupted" -- albeit at known points -- you have to consider
the possibility of the device *staying* in one of those intermediate
states indefinitely!)

This also has to address the fact that the user may *not* have applied
all of the upgrades that you require. I.e., you need to account for
how to get that device to a state (*the* state?) that your current
set of upgrades expects/requires.

And, throughout it all, you have to make sure that the upgrade
process itself continues to function flawlessly. (if you brick the
user's device, he won't be happy!)

The actual mechanics of creating an updatable framework are
conceptually pretty simple: finely *define* modules that are
*small* enough and "independant" enough that they can be replaced
individually.

As I said above, the bigger problem is accurately tracking the device's
current configuration and *knowing* what behavior you (and the user)
can expect from the device in *all* of those possible states.
(i.e., imagine a user who got to A.7 but never took the final step
to B.0. That user calls you complaining that his device is exhibiting
some "unusual behavior". You need to be able to recognize that the
device is at A.7 and needs to progress to B.0 to be in a "supported"
configuration)

On Fri, 20 May 2011 08:15:17 -0700, Tim Wescott wrote:

> Under windows you'd do this with dll's. Even though I use Linux, I
> can't remember the name of the same concept, but the file suffix is .so.

The formal term is "dynamic shared object" or "DSO", but most people seem
to call them "shared libraries" or "dynamic libraries" (or occasionally
even "DLLs", if they've been using Windows too long).

On May 20, 8:15*am, Tim Wescott <t...@seemywebsite.com> wrote:
> On 05/20/2011 04:17 AM, alacky3 wrote:
>
> > Hello.
>
> > I am implementing a program to be updated via GPRS channel. The total size
> > of the firmware reaches about 256 kB which makes it somewhat costly
> > upgrades. I considered the idea of dividing the firmware into independent
> > parts that can be updated independently and upgrade only the changed parts,
> > but I don't know how to begin. Does anyone has had a similar problem? How
> > do you have solved it?
>
> Under windows you'd do this with dll's. *Even though I use Linux, I
> can't remember the name of the same concept, but the file suffix is .so.

It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
(dll) for Shared Objects (so). However, there are quite a bit of run-
time overheads using them. For firmware, it might be easier to just
separate the library into "ROM" code and calls, traps or soft intr.
into it.

On 05/20/2011 10:27 AM, linnix wrote:
> On May 20, 8:15 am, Tim Wescott<t...@seemywebsite.com> wrote:
>> On 05/20/2011 04:17 AM, alacky3 wrote:
>>
>>> Hello.
>>
>>> I am implementing a program to be updated via GPRS channel. The total size
>>> of the firmware reaches about 256 kB which makes it somewhat costly
>>> upgrades. I considered the idea of dividing the firmware into independent
>>> parts that can be updated independently and upgrade only the changed parts,
>>> but I don't know how to begin. Does anyone has had a similar problem? How
>>> do you have solved it?
>>
>> Under windows you'd do this with dll's. Even though I use Linux, I
>> can't remember the name of the same concept, but the file suffix is .so.
>
> It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
> (dll) for Shared Objects (so). However, there are quite a bit of run-
> time overheads using them. For firmware, it might be easier to just
> separate the library into "ROM" code and calls, traps or soft intr.
> into it.

Yes, anything that's dynamically linking is going to be slow -- I was
trying to get some concepts and search terms to the guy as much as make
specific recommendations.

Certainly if size and speed were a concern, I'd want to do this with
something more statically linked (although I can't see how to avoid jump
tables or the equivalent). I can think of several ways to do this right
off the bat, but in the end you're going to have to sacrifice some
combination of speed, versatility, or design time.

--

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

>>> Under windows you'd do this with dll's. Even though I use Linux, I
>>> can't remember the name of the same concept, but the file suffix is .so.
>>
>> It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
>> (dll) for Shared Objects (so). However, there are quite a bit of run-
>> time overheads using them. For firmware, it might be easier to just
>> separate the library into "ROM" code and calls, traps or soft intr.
>> into it.
>
> Yes, anything that's dynamically linking is going to be slow -- I was
> trying to get some concepts and search terms to the guy as much as make
> specific recommendations.

I think (not sure) the shared library approach is slow only the first time
the library is demand loaded. After that, use of the library is the same as
if it was statically linked. Isn't that the extent of the slowness of shard
libraries?

I'm not sure of the memory resource usage at load time. I'll bet it's
temporary (extra memory released after loading happens).

JJS

alacky3 wrote:

> Hello.
>
> I am implementing a program to be updated via GPRS channel. The total size
> of the firmware reaches about 256 kB which makes it somewhat costly
> upgrades. I considered the idea of dividing the firmware into independent
> parts that can be updated independently and upgrade only the changed parts,
> but I don't know how to begin. Does anyone has had a similar problem? How
> do you have solved it?

The 256kB is not much; looks like a monolitic application to me.
Breaking it into the chunks would be difficult and inconvenient.

The good way to deal with this problem is DIFF. I.e. a compressed format
which uses the current application as a dictionary for the compression.
Only the references to the current dictionary and the changes have to be
transmitted.


Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

On Fri, 20 May 2011 11:19:19 -0700, John Speth wrote:

> I think (not sure) the shared library approach is slow only the first time
> the library is demand loaded. After that, use of the library is the same as
> if it was statically linked. Isn't that the extent of the slowness of shard
> libraries?

The global offset table (GOT) and procedure linkage table (PLT) have to be
generated each time a program is executed. The use of position-independent
code means that relocations don't have to be performed within the actual
code. The PLT can either be generated at startup or be generated lazily,
i.e. each entry is generated the first time that the function is called.
The GOT is always generated at startup.

Linking against static libraries results in the use of fixed addresses.
This reduces startup time, as no GOT/PLT entries are required, and
increases execution speed slightly (a typical estimate is 4%) as a level
of indirection is removed. The main performance downside is that, if
multiple processes are running different programs which use the same
library, each program will have its own copy of the library, which
increases the demand for memory.