1. This forum section is a read-only archive which contains old newsgroup posts. If you wish to post a query, please do so in one of our main forum sections (here). This way you will get a faster, better response from the members on Motherboard Point.

Firmware architecture for firmware update

Discussion in 'Embedded' started by alacky3, May 20, 2011.

  1. alacky3

    alacky3 Guest

    Hello.

    I am implementing a program to be updated via GPRS channel. The total siz
    of the firmware reaches about 256 kB which makes it somewhat costl
    upgrades. I considered the idea of dividing the firmware into independen
    parts that can be updated independently and upgrade only the changed parts
    but I don't know how to begin. Does anyone has had a similar problem? Ho
    do you have solved it?

    Thanks.



    ---------------------------------------
    Posted through http://www.EmbeddedRelated.com
     
    alacky3, May 20, 2011
    #1
    1. Advertising

  2. alacky3

    Tim Wescott Guest

    On 05/20/2011 04:17 AM, alacky3 wrote:
    > Hello.
    >
    > I am implementing a program to be updated via GPRS channel. The total size
    > of the firmware reaches about 256 kB which makes it somewhat costly
    > upgrades. I considered the idea of dividing the firmware into independent
    > parts that can be updated independently and upgrade only the changed parts,
    > but I don't know how to begin. Does anyone has had a similar problem? How
    > do you have solved it?


    Under windows you'd do this with dll's. Even though I use Linux, I
    can't remember the name of the same concept, but the file suffix is .so.
    At any rate, whether your OS supports it or not you'd need to have
    nice logical modules well separated by function (so that upgrading one
    thing doesn't break all the rest) and a means of dynamically linking the
    separate bits.

    --

    Tim Wescott
    Wescott Design Services
    http://www.wescottdesign.com

    Do you need to implement control loops in software?
    "Applied Control Theory for Embedded Systems" was written for you.
    See details at http://www.wescottdesign.com/actfes/actfes.html
     
    Tim Wescott, May 20, 2011
    #2
    1. Advertising

  3. alacky3

    Marc Jet Guest

    256KB on GPRS is not excessive. It will finish in less than 4 minutes
    worst case (2G single timeslot).

    If you really need faster updates or less bandwidth usage, you could
    inspire with RSYNC. It improves throughput for files of which a
    "similar" version is already present on the other side of the link.
    What definition of "similar" works best for your application, depends
    on the details of your project.

    You can even improve a lot over RSYNC, if you know exactly what's on
    the other side (by firmware version identifier). You could then
    implement a "patch processor" (interpreter with defined command set),
    and produce a patch instruction stream, which you then optimize,
    compress and transmit. On the other side of the link, it is
    decompressed and executed, and patches the existing firmware into the
    new version. Done right, this is the quickest/shortest possible way
    of transmitting the update (because it is a compression algorithm
    optimized to the specific application at hand).

    However, I suspect that standard compression algorithms on the binary
    firmware image and possibly a recommendation of using better wireless
    technology (EDGE or 3G) is quicker to deploy than either of these, and
    the user will not notice much of a difference anyway.
     
    Marc Jet, May 20, 2011
    #3
  4. alacky3

    D Yuniskis Guest

    On 5/20/2011 4:17 AM, alacky3 wrote:

    > I am implementing a program to be updated via GPRS channel. The total size
    > of the firmware reaches about 256 kB which makes it somewhat costly
    > upgrades. I considered the idea of dividing the firmware into independent
    > parts that can be updated independently and upgrade only the changed parts,
    > but I don't know how to begin. Does anyone has had a similar problem? How
    > do you have solved it?


    Presumably, you don't have a "spare" 256KB in the device to
    cache the image(s) -- persistently -- until *all* of them
    can be downloaded?

    If you try to do this piecemeal, you have potentially several
    hazards to avoid -- all having to do with incomplete or partially
    complete images being available.

    First, you have to have AT LEAST enough "cache" (though *this* can be
    volatile) to store the largest "piece" that you will be downloading.
    This safeguards against the case wherein the image can't be downloaded
    intact in a single piece (i.e., you can't risk "updating *while*
    downloading" for fear that the image is corrupted or interrupted
    *during* the download -- you need the previous "working" image
    to be available for your device to continue functioning in the
    event that the image isn't downloaded completely/correctly).

    You also need to have some strict sense of configuration management
    that allows you (your device and/or the device serving up these
    images) to know what the device in question's *current* configuration
    is and which "pieces" can be applied to that configuration to yield
    "operational" (albeit possibly with other "known bugs") configurations.

    For example, if the user is at version A.0, possible upgrade
    paths may be A.0->A.1->A.3->A.7->B.0, or A.0->A.2->A.7->B.0
    depending on which "modules" are updated along the way. Note
    that the path taken will determine which functionality (and
    "bug-onality") the device retains *while* it transitions to the
    final state (recall that the process may take minutes, hours,
    weeks, months or YEARS! Since you are allowing it to be
    "interrupted" -- albeit at known points -- you have to consider
    the possibility of the device *staying* in one of those intermediate
    states indefinitely!)

    This also has to address the fact that the user may *not* have applied
    all of the upgrades that you require. I.e., you need to account for
    how to get that device to a state (*the* state?) that your current
    set of upgrades expects/requires.

    And, throughout it all, you have to make sure that the upgrade
    process itself continues to function flawlessly. (if you brick the
    user's device, he won't be happy!)

    The actual mechanics of creating an updatable framework are
    conceptually pretty simple: finely *define* modules that are
    *small* enough and "independant" enough that they can be replaced
    individually.

    As I said above, the bigger problem is accurately tracking the device's
    current configuration and *knowing* what behavior you (and the user)
    can expect from the device in *all* of those possible states.
    (i.e., imagine a user who got to A.7 but never took the final step
    to B.0. That user calls you complaining that his device is exhibiting
    some "unusual behavior". You need to be able to recognize that the
    device is at A.7 and needs to progress to B.0 to be in a "supported"
    configuration)
     
    D Yuniskis, May 20, 2011
    #4
  5. alacky3

    Nobody Guest

    On Fri, 20 May 2011 08:15:17 -0700, Tim Wescott wrote:

    > Under windows you'd do this with dll's. Even though I use Linux, I
    > can't remember the name of the same concept, but the file suffix is .so.


    The formal term is "dynamic shared object" or "DSO", but most people seem
    to call them "shared libraries" or "dynamic libraries" (or occasionally
    even "DLLs", if they've been using Windows too long).
     
    Nobody, May 20, 2011
    #5
  6. alacky3

    linnix Guest

    On May 20, 8:15 am, Tim Wescott <> wrote:
    > On 05/20/2011 04:17 AM, alacky3 wrote:
    >
    > > Hello.

    >
    > > I am implementing a program to be updated via GPRS channel. The total size
    > > of the firmware reaches about 256 kB which makes it somewhat costly
    > > upgrades. I considered the idea of dividing the firmware into independent
    > > parts that can be updated independently and upgrade only the changed parts,
    > > but I don't know how to begin. Does anyone has had a similar problem? How
    > > do you have solved it?

    >
    > Under windows you'd do this with dll's.  Even though I use Linux, I
    > can't remember the name of the same concept, but the file suffix is .so.


    It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
    (dll) for Shared Objects (so). However, there are quite a bit of run-
    time overheads using them. For firmware, it might be easier to just
    separate the library into "ROM" code and calls, traps or soft intr.
    into it.
     
    linnix, May 20, 2011
    #6
  7. alacky3

    Tim Wescott Guest

    On 05/20/2011 10:27 AM, linnix wrote:
    > On May 20, 8:15 am, Tim Wescott<> wrote:
    >> On 05/20/2011 04:17 AM, alacky3 wrote:
    >>
    >>> Hello.

    >>
    >>> I am implementing a program to be updated via GPRS channel. The total size
    >>> of the firmware reaches about 256 kB which makes it somewhat costly
    >>> upgrades. I considered the idea of dividing the firmware into independent
    >>> parts that can be updated independently and upgrade only the changed parts,
    >>> but I don't know how to begin. Does anyone has had a similar problem? How
    >>> do you have solved it?

    >>
    >> Under windows you'd do this with dll's. Even though I use Linux, I
    >> can't remember the name of the same concept, but the file suffix is .so.

    >
    > It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
    > (dll) for Shared Objects (so). However, there are quite a bit of run-
    > time overheads using them. For firmware, it might be easier to just
    > separate the library into "ROM" code and calls, traps or soft intr.
    > into it.


    Yes, anything that's dynamically linking is going to be slow -- I was
    trying to get some concepts and search terms to the guy as much as make
    specific recommendations.

    Certainly if size and speed were a concern, I'd want to do this with
    something more statically linked (although I can't see how to avoid jump
    tables or the equivalent). I can think of several ways to do this right
    off the bat, but in the end you're going to have to sacrifice some
    combination of speed, versatility, or design time.

    --

    Tim Wescott
    Wescott Design Services
    http://www.wescottdesign.com

    Do you need to implement control loops in software?
    "Applied Control Theory for Embedded Systems" was written for you.
    See details at http://www.wescottdesign.com/actfes/actfes.html
     
    Tim Wescott, May 20, 2011
    #7
  8. alacky3

    John Speth Guest

    >>> Under windows you'd do this with dll's. Even though I use Linux, I
    >>> can't remember the name of the same concept, but the file suffix is .so.

    >>
    >> It's either Dynamic Linking Loader (dll) or Dynamic Loading Linker
    >> (dll) for Shared Objects (so). However, there are quite a bit of run-
    >> time overheads using them. For firmware, it might be easier to just
    >> separate the library into "ROM" code and calls, traps or soft intr.
    >> into it.

    >
    > Yes, anything that's dynamically linking is going to be slow -- I was
    > trying to get some concepts and search terms to the guy as much as make
    > specific recommendations.


    I think (not sure) the shared library approach is slow only the first time
    the library is demand loaded. After that, use of the library is the same as
    if it was statically linked. Isn't that the extent of the slowness of shard
    libraries?

    I'm not sure of the memory resource usage at load time. I'll bet it's
    temporary (extra memory released after loading happens).

    JJS
     
    John Speth, May 20, 2011
    #8
  9. alacky3 wrote:

    > Hello.
    >
    > I am implementing a program to be updated via GPRS channel. The total size
    > of the firmware reaches about 256 kB which makes it somewhat costly
    > upgrades. I considered the idea of dividing the firmware into independent
    > parts that can be updated independently and upgrade only the changed parts,
    > but I don't know how to begin. Does anyone has had a similar problem? How
    > do you have solved it?


    The 256kB is not much; looks like a monolitic application to me.
    Breaking it into the chunks would be difficult and inconvenient.

    The good way to deal with this problem is DIFF. I.e. a compressed format
    which uses the current application as a dictionary for the compression.
    Only the references to the current dictionary and the changes have to be
    transmitted.


    Vladimir Vassilevsky
    DSP and Mixed Signal Design Consultant
    http://www.abvolt.com
     
    Vladimir Vassilevsky, May 20, 2011
    #9
  10. alacky3

    Nobody Guest

    On Fri, 20 May 2011 11:19:19 -0700, John Speth wrote:

    > I think (not sure) the shared library approach is slow only the first time
    > the library is demand loaded. After that, use of the library is the same as
    > if it was statically linked. Isn't that the extent of the slowness of shard
    > libraries?


    The global offset table (GOT) and procedure linkage table (PLT) have to be
    generated each time a program is executed. The use of position-independent
    code means that relocations don't have to be performed within the actual
    code. The PLT can either be generated at startup or be generated lazily,
    i.e. each entry is generated the first time that the function is called.
    The GOT is always generated at startup.

    Linking against static libraries results in the use of fixed addresses.
    This reduces startup time, as no GOT/PLT entries are required, and
    increases execution speed slightly (a typical estimate is 4%) as a level
    of indirection is removed. The main performance downside is that, if
    multiple processes are running different programs which use the same
    library, each program will have its own copy of the library, which
    increases the demand for memory.
     
    Nobody, May 20, 2011
    #10
  11. alacky3

    larwe Guest

    On May 20, 11:56 am, Marc Jet <> wrote:
    > 256KB on GPRS is not excessive.  It will finish in less than 4 minutes
    > worst case (2G single timeslot).


    I wish this was true. In real world conditions I can tell you from
    bitter personal experience that it is more like two hours
    "worst" [observed] case with an average more like 30-45 minutes.
    That's for a gzipped image that was originally 256K pre-compression,
    too. Tested in many many sites.
     
    larwe, May 21, 2011
    #11
  12. larwe skrev 2011-05-21 04:33:
    > On May 20, 11:56 am, Marc Jet<> wrote:
    >> 256KB on GPRS is not excessive. It will finish in less than 4 minutes
    >> worst case (2G single timeslot).

    >
    > I wish this was true. In real world conditions I can tell you from
    > bitter personal experience that it is more like two hours
    > "worst" [observed] case with an average more like 30-45 minutes.
    > That's for a gzipped image that was originally 256K pre-compression,
    > too. Tested in many many sites.
    >


    Normally, you pay per bit and not per minute,
    so is the time important?
    In Sweden, to download 256 kB would cost about $0.50.

    BR
    Ulf Samuelsson
     
    Ulf Samuelsson, May 21, 2011
    #12
  13. Nobody wrote:
    > On Fri, 20 May 2011 11:19:19 -0700, John Speth wrote:
    >>I think (not sure) the shared library approach is slow only the first time
    >>the library is demand loaded. After that, use of the library is the same as
    >>if it was statically linked. Isn't that the extent of the slowness of shard
    >>libraries?

    >
    > The global offset table (GOT) and procedure linkage table (PLT) have to be
    > generated each time a program is executed. The use of position-independent
    > code means that relocations don't have to be performed within the actual
    > code. The PLT can either be generated at startup or be generated lazily,
    > i.e. each entry is generated the first time that the function is called.
    > The GOT is always generated at startup.


    The thread starter was talking about a firmware image of 256 kByte
    total. I doubt he's using a full off-the-book System V shared object
    implementation. There's plenty of room for optimisation.

    For example, the GOT and PLT can be generated at the time of the
    firmware update. The shared object can be stored in fully-relocated
    form. It doesn't have to use position-independant code if we can patch
    the binary code. Essentially, this is what MS Windows does. It's called
    "binding":
    http://blogs.msdn.com/b/oldnewthing/archive/2010/03/18/9980802.aspx

    > Linking against static libraries results in the use of fixed addresses.
    > This reduces startup time, as no GOT/PLT entries are required, and
    > increases execution speed slightly (a typical estimate is 4%) as a level
    > of indirection is removed.


    This is what happens when you load a Linux kernel module. Up to kernel
    2.4.x, a module is a standard .o file, which is statically linked
    against the running kernel (in 2.6 it's slightly different but the basic
    process, as far as I know, is the same).


    Stefan
     
    Stefan Reuther, May 21, 2011
    #13
  14. alacky3

    larwe Guest

    On May 21, 12:19 am, Ulf Samuelsson <> wrote:

    > > I wish this was true. In real world conditions I can tell you from
    > > bitter personal experience that it is more like two hours
    > > "worst" [observed] case with an average more like 30-45 minutes.

    >
    > Normally, you pay per bit and not per minute,
    > so is the time important?


    Sure, it is per-byte payment here in the USA also (except for lucky
    people like me who kept some old iPhone account SIMs active to be
    grandfathered into unlimited data ;) but the problem is downtime, and
    sequencing.

    Downtime can be removed if you have enough space to store a redundant
    image, but the sequencing/scheduling issue is very important. Our
    equipment for instance may need for legal reasons to have a technician
    must be on site when the upgrade is rebooted to test the system and
    certify it operation. The ideal scenario would be for tech to arrive,
    push "upgrade", wait for the code to arrive (a few minutes is ok),
    then test the system and leave. Since the time is so unpredictable and
    always LONG, this workflow doesn't work.

    Also bear in mind we are deploying to hundreds of thousands, if not
    millions of units.
     
    larwe, May 21, 2011
    #14
  15. alacky3

    David Brown Guest

    On 20/05/11 13:17, alacky3 wrote:
    > Hello.
    >
    > I am implementing a program to be updated via GPRS channel. The total size
    > of the firmware reaches about 256 kB which makes it somewhat costly
    > upgrades. I considered the idea of dividing the firmware into independent
    > parts that can be updated independently and upgrade only the changed parts,
    > but I don't know how to begin. Does anyone has had a similar problem? How
    > do you have solved it?
    >
    > Thanks.
    >


    Can you separate code and data? Maybe the 256K contains tables, images,
    fonts, or something that is relatively fixed, and can be separated.
    It's a lot easier to add a little indirection to the access of such
    tables than to have more general dynamically linked libraries.

    Can you patch the firmware, so that the user downloads a delta from one
    version to the next?
     
    David Brown, May 21, 2011
    #15
  16. In article <>,
    alacky3 <sr.alacran@n_o_s_p_a_m.gmail.com> wrote:
    >Hello.
    >
    >I am implementing a program to be updated via GPRS channel. The total size
    >of the firmware reaches about 256 kB which makes it somewhat costly
    >upgrades. I considered the idea of dividing the firmware into independent
    >parts that can be updated independently and upgrade only the changed parts,
    >but I don't know how to begin. Does anyone has had a similar problem? How
    >do you have solved it?


    A straightforward solution is to have two sections for each library.
    One section contains addresses of subroutines in the other section.
    The main program and the libraries are linked to the section with
    the addresses.
    The linking can be difficult, but I blame those linkers,
    because this is conceptually simple.

    Each pair of sections must be put in a fixed place.
    A pair of sections can be exchanged without affecting linking.
    The overhead is very small, an extra indirection.

    In assembler it is a snap, especially with macro's.
    In c it looks bad.
    An instrumented c-compiler could handle it transparently.

    The details are much dependant on the tools you have available.

    >Thanks.


    Groetjes Albert

    --
    --
    Albert van der Horst, UTRECHT,THE NETHERLANDS
    Economic growth -- being exponential -- ultimately falters.
    albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
     
    Albert van der Horst, Jun 1, 2011
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Tim

    4600 Memory Architecture

    Tim, Feb 6, 2004, in forum: Dell
    Replies:
    4
    Views:
    344
  2. Replies:
    0
    Views:
    351
  3. Arun Prasad
    Replies:
    0
    Views:
    2,383
    Arun Prasad
    Aug 18, 2003
  4. Muhammad Khan
    Replies:
    3
    Views:
    471
    Ken McElvain
    Jul 10, 2003
  5. nosredna
    Replies:
    5
    Views:
    311
    nosredna
    Mar 8, 2005
Loading...

Share This Page