1. This forum section is a read-only archive which contains old newsgroup posts. If you wish to post a query, please do so in one of our main forum sections (here). This way you will get a faster, better response from the members on Motherboard Point.

ELF file: linking in large binary data

Discussion in 'Embedded' started by Will Barry, Oct 9, 2004.

  1. Will Barry

    Will Barry Guest

    Hi,

    I would like to include a large block (100+KB)of pre-defined binary
    data into my executable. Mainly, I will be storing static bitmap
    images.
    Any ideas on how to do it? and what tools are necessary?

    Basically, what I would like to do is make an all-purpose resource
    object file containing various binary data. Such as the resource dlls
    in MS Windows.

    Btw, my cpu is an ARM and I'm using the ARM tools.

    Thanks,
    Will Barry
     
    Will Barry, Oct 9, 2004
    #1
    1. Advertisements

  2. Will Barry

    WaldemarIII Guest

    Convert your binary image to whatever data structure you
    like (in human readable form, mind you), compile, link and
    presto... That's what I do, whenever the use of files is
    undesirable if not impossible.

    BCNU

    Waldemar
     
    WaldemarIII, Oct 9, 2004
    #2
    1. Advertisements

  3. armasm will let you include binary files using INCBIN.

    Wilco
     
    Wilco Dijkstra, Oct 9, 2004
    #3
  4. Will Barry

    Will Barry Guest

    Convert your binary image to whatever data structure you
    Thanks! But the file I would like to link with the executable is
    really large, like 100K+ or so. So converting manually is out of the
    question. But then writing the program to generate the binary data
    into a pre-initialized C array form would not be difficult. This is
    going to be my fallback if I can't find an easier way. I appreciate
    your reply!

    Will Barry
     
    Will Barry, Oct 12, 2004
    #4
  5. Will Barry

    Will Barry Guest

    I would like to include a large block (100+KB)of pre-defined binary
    Thanks! Also, fyi, I took the liberty of searching for your keywords
    'armasm' and 'INCBIN' in google. I found a link in the arm.com
    website regarding the utility called bin2aof. This program converts
    any binary file into a format that can be linked in with the
    executable.

    Thanks again.
    Will =>barry
     
    Will Barry, Oct 12, 2004
    #5
  6. Will Barry

    CBFalconer Guest

    Why bother? Just let the executable read the file into an array in
    its own data space on startup. Much more flexible, and no
    opportunity for translation foulups.
     
    CBFalconer, Oct 12, 2004
    #6
  7. Read the file from where? None of the embedded systems I've
    ever worked on had filesystems.
    But it requires a filesystem, which is a huge amount of
    overhead when all you want is an initialized data structure.
     
    Grant Edwards, Oct 12, 2004
    #7
  8. Will Barry

    Barry Will Guest

    Thanks for all the replies.
    That is correct, I don't have a filesystem. So I guess the only place
    to store the binary data is together with the executable. I'm
    currently playing around with the INCBIN armasm directive which Wilco
    suggested. According to the documentation, this directive links a raw
    binary file into the generated object file as is, with no
    interpretation. This looks like an easy way to do it. I'm gonna try
    it and see how it works...
     
    Barry Will, Oct 13, 2004
    #8
  9. Will Barry

    CBFalconer Guest

    Most systems have some sort of rudimentary filesystem, even if it
    is not recognizable as such. Especially systems that need 100k of
    raw data. What is the system that transfers that load module into
    runnable code space, and what is used to implement stdin and stdout
    (if any). If the system is doing any user interaction, it has a
    file system.
     
    CBFalconer, Oct 13, 2004
    #9
  10. Huh? I've been doing embedded systems stuff for 25 years and
    have never worked on anything that had any sort of filesystem
    (even rudimentary).
    Huh? I burn my "load modules" into ROM.
    Nothing! There is no filesystem. There sometimes is a
    function I can call to write a string to a UART, but it's not a
    filesystem.
    That's complete nonsense.
     
    Grant Edwards, Oct 14, 2004
    #10
  11. Most systems have some sort of rudimentary filesystem, even if it
    is not recognizable as such. Especially systems that need 100k of
    raw data. What is the system that transfers that load module into
    runnable code space, and what is used to implement stdin and stdout[/QUOTE]

    Lets see one system transfers and verifies 640KB into real time video
    run space. Another transfers 128KB into a different form of dual
    ported RAM, both use DMA.
    One system has a tracking camera and a rarely used toggle switch for
    user interaction.

    Another has 6 LEDs and two touch switches.

    Both have a 16bit code set in the 64KB to 128KB size and NO file I/O
    especially in NO way related to user I/O.

    Debug has a very rudimentary Command Line Interpreter, with no file i/o
    system there.
     
    Paul Carpenter, Oct 14, 2004
    #11
  12. I always use that technique, myself. I usually do it in python or Matlab,
    rather than C, because that's easier to tweak and still easy to make part
    of the build process (make files). It also has the advantage that it's
    easy to perform numerical-domain transformations on the data on the way,
    like quantization and rounding.

    However, since you're dealing with images, it might interest you to know
    that the original default PNM (portable-any-map) or pbm image
    transformation library format was in fact syntactically a C array
    initializer. That might be a convenient pre-built tool for the job. Lots
    of original X-windows programs compiled all of their icons in, even though
    they would most likely be running on machines with file systems, to avoid
    the problem of ensuring that the path to the directory containing the
    icons/images was properly configured. So there's actually plenty of
    experience out there in existing code...

    I still don't understand why some object file formats (COFF?) are so
    baroque about these things, though. On one platform that I've used, the
    COFF image of a constant initialized array was several orders of magnitude
    larger than the original...

    Cheers,
     
    Andrew Reilly, Oct 14, 2004
    #12
  13. Will Barry

    ammonton Guest

    Have you looked at using GNU objcopy to create object files from your
    binaries?

    -a
     
    ammonton, Oct 14, 2004
    #13
  14. It seems that you have been infected by the rotten Unix I/O ideology
    for a too long time :) :).

    So apparently any sequence of two or more bytes would be a file
    system. Thus, a 16-32 bit processor with en 8 bit data bus (such as
    68008) would have a file system since multiple bytes are transferred
    in sequence to get a word transfer.

    Why not take the idea even further and define that any sequence of
    bits is a file system. Thus, even a UART sending _only_ the SYN
    character at regular intervals would have a file system.
    On an embedded system, that systems that transfers the load module is
    often a human hand moving the EPROM from the prommer to the embedded
    system PROM socket.
    What is the point of having these channels, if you do not have
    anywhere to connect them.
    So a system that only reads a switch (on a safety gate that shuts down
    the system, if someone tries to enter a hazardous area) is a file
    system ?

    Paul
     
    Paul Keinanen, Oct 14, 2004
    #14
  15. We don't need to do that --- there already are enough people
    (including some vintage CS professors) who go around stating that
    "everything is a database", and they actually mean it. Classic case
    of new hammer syndrome, if you ask me.
     
    Hans-Bernhard Broeker, Oct 14, 2004
    #15
  16. Will Barry

    Ben Jackson Guest

    The most portable way of doing this is to write a C program that
    generates a C array declaration. This is in fact the exact format
    of the X11 bitmap and pixmap files.

    You really don't want to mess with ELF by hand (assuming your
    toolchain builds an ELF object which you process with something like
    `objcopy -O binary'). ELF is arcane beyond belief.

    If your data is Too Big for that (but 100KB isn't that bad) I'd make
    a stub C file that builds an array with a magic number that you can
    find and overwrite:

    char data[SIZE] = { 'm', 'a', 'g', 'i', 'c' };

    The magic serves two purposes: That's how you find it with your
    data-stamping program, and also it forces the array into data space
    so you don't just get a bunch of ELF gunk and no array.
     
    Ben Jackson, Oct 14, 2004
    #16
  17. Will Barry

    CBFalconer Guest

    Well, in the past my systems consisted of code that interacted with
    the outside world in some manner. They had no disk file systems,
    but they did read switch contacts, A/D converters and the ilk, and
    they output information to such things as leds, even displays,
    lans, etc. All those things appeared as the specialized file
    system for that particular application. The connections were
    implemented as drivers for the devices. Devices could be brought
    on (and off) line (connected to the interrupt system) by opening
    (and closing) files.

    This was a great convenience, since I could capture a run of the
    device to actual disk files, and re-run any problems indefinitely,
    measure response times, etc. ad nauseam.

    I suggest you stop thinking of files in such a limited manner, but
    rather as the communication path between the world and your
    software. Without such the software is totally useless. By
    abstracting that interface you will have many more abilities.
     
    CBFalconer, Oct 14, 2004
    #17
  18. Will Barry

    David Brown Guest

    Such an arrangement can be very useful - that's one of the nice things about
    working with a *nix system. It is also possible to make such an abstraction
    even without a *nix OS. But no matter how it is done, it takes a lot of
    overhead. In "big" embedded systems, it is probably worth it for the
    flexibility and convenience, while in "small" systems it is not. So while
    most of the systems *you* have worked with have some sort of filesystem, I
    think most of the systems used regularly by people in this group have no
    filesystem of any sort.
     
    David Brown, Oct 14, 2004
    #18
  19. That's a very, uh, unique way of doing things. I've never
    heard of anybody implimenting a filesystem so they can read a
    pushbutton switch or turn an LED on/off before.
    I guess I don't see how being able to open/close an I/O device
    makes it into a filesystem.
    So an LED is a filesystem. You've redefined the word
    filesystem so broadly that it's become meaningless.
     
    Grant Edwards, Oct 14, 2004
    #19
  20. Will Barry

    CBFalconer Guest

    Which doesn't make it any less true, just as every wire is a
    transmission line. The problem is to know when to apply the
    simpler approximations. Similarly, I suspect very few ships adjust
    for relativaty path distortion in their navigation, although the
    GPS satellites very well may do so.
     
    CBFalconer, Oct 14, 2004
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.