1. This forum section is a read-only archive which contains old newsgroup posts. If you wish to post a query, please do so in one of our main forum sections (here). This way you will get a faster, better response from the members on Motherboard Point.

load binary modules at run time

Discussion in 'Embedded' started by FabiusAone, May 21, 2014.

  1. FabiusAone

    FabiusAone Guest

    Hi guys,

    I have 100 c modules, my system has to run only one module at a time (at run time); these modules have few functions called by the system and these have the same prototipe for every module. These modules has a lot of dependencies with the system.
    My first approach was create only one project with all files, assign one section for each module, specify that all these sections must run in the same section.

    Example: module A, B

    name=".text_A" runin=".text_runmodule"
    name=".rodata_A" runin=".rodata_runmodule"
    name=".data_A" runin=".data_runmodule"
    name=".bss_A" runin=".bss_runmodule"

    name=".text_B" runin=".text_runmodule"
    name=".rodata_B" runin=".rodata_runmodule"
    name=".data_B" runin=".data_runmodule"
    name=".bss_B" runin=".bss_runmodule"

    with objcopy I extracted all the sections so I can create A.bin a B.bin e load these files at run time on the runmodule sections (in RAM)

    I created an array of a structure that contains pointer to the module's functions.

    drawbacks are:
    1 - I have to create a lot of sections
    2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
    3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..

    Any suggestion? Static or shared library? What else?..

    FabiusAone, May 21, 2014
    1. Advertisements

  2. FabiusAone

    Tauno Voipio Guest

    This is the classical overlay structure. Look in the linker/loader
    manual for 'Overlay loading'. Google may also help.
    Tauno Voipio, May 21, 2014
    1. Advertisements

  3. FabiusAone

    Stef Guest

    In comp.arch.embedded,
    My first thought was "overlays". Used mostly one or two decades ago, I think.
    But may be a nice term to search for. ;-)
    Stef, May 21, 2014
  4. FabiusAone

    David Brown Guest

    What sort of tools are you using? "objcopy" implies gcc. But "create
    one project" implies that you are using an IDE to handle the build
    process. My thoughts here are that gcc, ln, objcopy, and make can
    combine to produce magic, but you will have to learn the joys of make
    and Makefiles. It can be hard going, and debugging Makefiles is "fun",
    but it should be possible to get a build system together so that when
    you change a file, all dependent modules get rebuilt as necessary, but
    nothing else.

    There are a couple of shortcuts that can make a big difference for large
    builds, whether you are using your own Makefile or an IDE. First, make
    sure you have parallel build enabled with an appropriate number of jobs
    for the number of cores in your PC. Second, try to use ccache to make
    redundant compiles a little faster.

    The other thing you should probably do is decouple your modules and your
    base system. Arrange for your modules to have an entry point (or entry
    points) at a fixed address at the start of the module. This entry point
    may register other functions with the base system, depending on the
    flexibility you need. Functions from the base system that are needed by
    the modules should be exported as an array of pointers, and the address
    of that array should be passed to the module via its entry function.
    Once you have this sort of system in place, the base system and the
    modules can be compiled and linked independently - there will be no need
    to update everything each time something changes in the base system.
    David Brown, May 21, 2014
  5. Thanks but...this is what I did..or not?

    What about drawbacks?:
    1 - I have to create a lot of sections
    2 - when I'm working with a module, I have to exclude from build the other modules to speed up time
    3 - every time I modify a module I have to recompile all the modules because system has changed..and theoretically I SHOULD retest all the modules..
    fabio.ferrari.aone, May 21, 2014
  6. FabiusAone

    FabiusAone Guest

    Hi David,
    I use CrossWorks (gcc) with ARM Cortex M4, parallel compile are enabled butit would be nice to have a base system that don't change each time you modify a module..
    Your "decoupling" approach works well regarding module's functions needed by the base system, in fact I think that I'll maintain this approach that I tried in these days, but regarding the needs of the modules there's a problem: low level dependencies..for example __float64_div..I have no control ofthese symbols..
    probably the best solution could be to force duplication of these pieces ofcode in each module..but I'm not able to do it..
    FabiusAone, May 21, 2014
  7. FabiusAone

    Randy Yates Guest

    What the hell are you trying to do? Even a smallish ARM Cortex M4 has
    512 KB of FLASH these days. Why are you trying to overlay?!?!?
    Randy Yates, May 21, 2014
  8. 2014-05-21 12:28, FabiusAone skrev:
    has changed..and theoretically I SHOULD retest all the modules..
    Why not map each module to a task ,and only run a single task at a time?

    You want a message passing RTOS.

    Ulf Samuelsson
    Ulf Samuelsson, May 21, 2014
  9. FabiusAone

    David Brown Guest

    (Please get yourself a newsreader and a newsserver, rather than the
    crappy Google Groups interface. Google Groups is fine for searching,
    but screws up the formatting of posts. Thunderbird and
    news.eternal-september.org are free and work on all platforms. And
    please don't top-post.)

    Make your modules separate projects, so that they are self-contained
    except for accessing specific functions in the base system via a jump
    table. They can even have a "main" function - this can be the entry
    point for the module. You will end up with duplication of library code
    between the base system and modules, but that is unlikely to matter - as
    long as you follow good practices of not using floating point
    (especially doubles) unless they are needed, not using things like
    printf(), etc., then the duplicated library code is going to be fairly
    small. If there are some functions that you know you need and you know
    they cause a lot of extra code, such as printf(), you can put them in
    the jump table. You can't necessarily do it with gcc library functions
    (as distinct from newlib or libc functions) - functions starting with
    two underscores are compiler-private functions and may have weird
    calling conventions that don't work when you try to use function
    pointers. And if you use malloc() and free(), you have to keep them in
    the base system rather than having two copies.

    David Brown, May 21, 2014
  10. FabiusAone

    Fabius Guest

    5MB..for now..
    Fabius, May 22, 2014
  11. FabiusAone

    David Brown Guest

    How do you possibly manage to get 5 MB of code? It's easy to use up
    lots of space with data, and equally easy to handle loading that at runtime.

    If you really need that much code, get a Cortex M4 with an external
    memory bus and use that.
    David Brown, May 22, 2014
  12. FabiusAone

    Randy Yates Guest

    Thanks David. I wanted to say this, but I was supposing Fabius has some
    very weird corner-case.
    Randy Yates, May 23, 2014
  13. FabiusAone

    David Brown Guest

    I'm sure he has good reasons for wanting this arrangement, but I think
    it could be worth his while taking a step back and telling us what he is
    looking for in the complete system. Then maybe he can get ideas here
    for alternative approaches that would work better.
    David Brown, May 23, 2014
  14. FabiusAone

    Fabius Guest

    Thanks to all of you, I'll try to explain better the situation, sorry
    for my english:

    I'm developing a system where the user can choose a set of
    functionalities grouped by c modules, there are no dependencies between
    these modules. The user can wait 1 second to obtain the functionality
    but I would a fast system startup.
    5 MB is the total sum of the binary modules for now, but I can load a
    single module on the internal SRAM without problem (I'd prefer keep
    secure this code.. ) from an SDcard where are stored (crypted).
    If I maintain only one project with all c modules thanks to the overlay
    method I'm able to develop this application..but I'm not happy that each
    time that I add o modify a module I have to recompile all the modules
    and the base system..I'd prefer a modular system..

    I'm using a Cortex M4 (without MMU) with Crossworks (gcc).
    Fabius, May 23, 2014
  15. FabiusAone

    Tauno Voipio Guest

    I do not quite understand the re-compile requirement. The
    system is so large that it is sensible to build it from
    modules, and you need to re-compile only those modules that
    have been changed. The make utility helps automating this

    You do have to link the overlay file again, if an overlay
    changes, but there is no need to change the root code, unless
    the calling sequence to the overlay code changes.
    Tauno Voipio, May 23, 2014
  16. FabiusAone

    upsidedown Guest

    Sounds like a classic overlay loading case.

    I don't know the features of the tool chain but assuming that the
    linker is capable of generating pure binary modules for a specific
    load address, things should be quite simple. Just decide a common load
    address for all your 100 object modules and compile and link them
    independently with that load address. No need for position independent
    code (PIC) or for fixups during loading.

    If you change one module, just recompile that module.

    If your modules use some common routines, such as C-runtime routines,
    put these into the root of the tree. Link this root and get a symbol
    table file from that linking. Link all your modules against this
    symbol table file. If you change the tree root, you need to generate a
    new symbol table file and relink all your 100 modules against it.

    If your module contains multiple entry point, link an indirect jump
    table in the beginning of each module, so you can recompile each
    module independently. I have used this trick, when programming a 4 to
    8 KiB EPROM took at least 5 minutes each, thus making a change to one
    routine took only 5 minutes to reprogram instead of using an hour to
    reprogram a dozen EPROMS.

    You should look at the documentation of your tool chain, if you can
    specify an absolute load address for compilation and linking and also
    verify that you can generate an absolute file (no relocation or
    fixups, no debug info) in file.

    You could also search for terms like "autoload vector" "RSX-11" (with
    quotes), to see how autoload was implemented in the past, but of with
    manual overlay loading, there were far more alternatives on a PDP-11.
    upsidedown, May 23, 2014
  17. FabiusAone

    Les Cargill Guest

    It looks a lot like you're using Linux. That probably
    means you have POSIX.

    The POSIX constellation has dlopen / dlsym / dlerror /
    dlclose to dynamically load modules. I've only used this with .so files,
    but it may go beyond that. The fail cases for this are *painful* but it

    When you do this, it might be worthwhile to get out of the IDE
    and use a console/shell prompt and makefiles.

    You'll have to more or less hand-code dependencies somewhere, but if
    you're rigorous with header files, you can use "gcc -M" to generate a
    file full of dependencies and munge that.

    You sound (downthread) like you *really* need a good regression
    suite derived from unit tests and integration tests.
    Les Cargill, May 24, 2014
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.