1. This forum section is a read-only archive which contains old newsgroup posts. If you wish to post a query, please do so in one of our main forum sections (here). This way you will get a faster, better response from the members on Motherboard Point.

LPC21XX and FreeRTOS - Debug, Timing, or Stack (?) Problem?

Discussion in 'Embedded' started by lawzaz, Jan 21, 2013.

  1. lawzaz

    lawzaz Guest

    Hi,

    I've adopted some old code and I'll be the first to admit I have no ide
    what I'm doing. I'm working with LPC2129-based hardware running FreeRTO
    6.1. My code executed until I made a few small changes (defining an
    printing an array plus some basic mathematical operations). Now, when
    read CAN values, "if" statements based on these values never trigge
    properly (almost as if the values change between printing and testing). M
    sample rate is slow, so I doubt this - but it is possible. I have als
    disabled interrupts.

    If I use DEBUG everything works perfectly. My DEBUG is defined as

    Code:
    #define DEBUG_puts UART0_puts
    #define DEBUG_putc UART0_putc
    void debug_printByte(uint8_t d) {
    if (((d & 0xF0) >> 4) < 0xA)
    debug_putc('0' + ((d & 0xF0) >> 4));
    else
    debug_putc('A' + (((d & 0xF0) >> 4)-0xA));
    if ((d & 0x0F) < 0xA)
    debug_putc('0' + (d & 0x0F));
    else
    debug_putc('A' + ((d & 0x0F)-0xA));
    }
    
    When DEBUG is disabled, it's:

    Code:
    #define debug_printf(x)
    #define debug_putc(x)
    #define debug_printByte(x)
    
    so nothing at all happens.

    If the problem only happens when DEBUG is disabled, how can I troubleshoo
    using only UART, since there's no other headers on the board? I coul
    potentially insert a delay... but why would I want to slow my code down
    and isn't that the point of an RTOS?

    The last time I had this issue, I increased the stack size for each tas
    and this solved the problem. This time, even a substantial stack increas
    will not result in functioning code.

    I'd be very appreciative of any help you can offer. I'm at a loss as to ho
    to troubleshoot, and it is unlikely that shots in the dark as to th
    problem will help when dealing with 30+ files and thousands of lines o
    possible code.

    Thanks!
     
    lawzaz, Jan 21, 2013
    #1
    1. Advertisements

  2. lawzaz

    Rich Webb Guest

    Stack size would have been my first guess but I see you've already
    tried that.

    I have seen some compilers that do register tracking for autos and
    sometimes get it wrong. That is, keep auto variables in registers
    rather than on the stack and reuse a register(s) for more than one
    auto variable within a function because it determined (incorrectly, in
    that case) that the prior occupant was not used again and the register
    was available. More likely to happen with the higher optimization in
    release versus debug mode, I'd imagine. Check your map files?
     
    Rich Webb, Jan 21, 2013
    #2
    1. Advertisements

  3. lawzaz

    lawzaz Guest

    Good thought, thank you - and I'll check the map - but my DEBUG isn't
    true bug. It's just a flag to enable printing. I believe my optimizatio
    using ARM-ELF-GCC is -OS.
     
    lawzaz, Jan 21, 2013
    #3
  4. Do you have anything that isn't marked as volatile but which should be ?

    I would also ask if you had anything which isn't atomic for updating or
    accessing and should have some form of a synchronisation guard around it
    but I see you have already disabled interrupts.

    Have you looked at the generated code for a faulty part of the code to
    see what is really going on ?

    Simon.
     
    Simon Clubley, Jan 21, 2013
    #4
  5. First, may I ask why you are posting your question here rather than in
    the rather excellent FreeRTOS support forum? You may find *some*
    FreeRTOS experts who can help here, on the FreeRTOS support forum you
    will find *lots*, and if they can't help you can get help direct from
    the authors (me, for example)
    http://www.freertos.org/FreeRTOS_Support_Forum_Archive/freertos_support_forum_archive_index.html

    Second, have you been through the FAQ "my application does not run, what
    could be wrong?"?
    http://www.freertos.org/FAQHelp.html


    -OS is very aggressive and handled much better in new versions of GCC.
    How old is your GCC version? Try setting down to -O1 to see if the
    problem goes away, and if it does, and you are using the latest GCC
    version, it is likely to be an subtle issue in your C code.

    Regards,
    Richard.

    + http://www.FreeRTOS.org
    Designed for microcontrollers. More than 103000 downloads in 2012.

    + http://www.FreeRTOS.org/plus
    Trace, safety certification, UDP/IP, TCP/IP, training, and more...
     
    FreeRTOS info, Jan 21, 2013
    #5
  6. lawzaz

    David Brown Guest

    (Top-posting fixed.)

    Unless you are using a truly ancient version of gcc, or a half-finished
    development version, or a weird patched version from some amateur, then
    it is /highly/ unlikely to be a bug in gcc. Like every tool, there
    /are/ bugs in gcc - but like every tool that is used by vast numbers of
    developers every day, it is very unlikely that you'll trigger a bug in
    "normal" code.

    It is more likely that your code is incorrect in some way.

    From the description of your code, it sounds like you have have written
    code that requires data to be accessed (read or written) to memory at a
    particular point in the code, but have not forced that in the code.
    When your debugging is enabled, the code leads to external function
    calls which force the compiler to write out data to memory before the
    call and read in after the call, if there is any way for the external
    function to access that data (and you are not using link-time
    optimisation). When optimising code, the compiler will not necessarily
    access memory variables in the order given in the source code - it may
    not access them at all if it can see the accesses are unnecessary.

    The most common cause of this, as another poster suggested, is missing
    "volatile" qualifiers on the variable definitions or the memory
    accesses. Another more subtle one is a misunderstanding of how volatile
    accesses intersperse with non-volatile accesses - many people think that
    volatile accesses enforce an ordering with ordinary accesses too, which
    is incorrect.

    One of the easiest ways to check this is to replace the debug calls with
    memory barriers (FreeRTOS may have a standard macro defined for this, or
    you can use "asm volatile ("" ::: "memory")" with gcc).
     
    David Brown, Jan 21, 2013
    #6
  7. lawzaz

    lawzaz Guest

    My reason for posting here is because I didn't know if the problem was
    with FreeRTOS or elsewhere! I imagine it has more to do with my lack of
    understanding of RTOS's in general, so I didn't want to clutter that forum
    when a more general forum may be just as good for me.

    I did some experimenting today, since I like to learn by doing and have no
    background in this stuff (computer science - I'm a mechanical engineer).
    A few tests, trying to back out where the problem might be.

    Stack/heap:
    A marginally larger stack - no change
    A marginally larger heap - no change
    A much larger stack - overflow

    Optimization:
    -O1, -O2, -0S same result (no optimization would not fit in flash)

    Volatility:
    After reading, it seems maybe my CAN packet response could be defined as
    volatile. Declaring my CAN packet variable volatile - no change

    Line-by-line debug:
    With DEBUG disabled, two functions are affected.
    A: reading a multi-frame CAN message, and
    B: an "if" statement, using the value from a CAN packet as a trigger

    I enabled DEBUG to get back to working condition and commented print
    until
    things broke.

    (bytes) (function A) (function B)
    54239 - !A, !B
    54273 - !A, B
    54304 - A, !B
    54358 - !A, B
    54438 - A, B

    It didn't seem to make a difference which lines were commented as much as
    what the filesize was (the "break it" lines were very basic and not invoke

    in the troublesome functions at all).

    Memory:
    Then, I tried to see if it was a problem with memory addresses b
    assigning

    int debug_printf(char *fmt, ...) { asm volatile ("" ::: "memory"); retur
    0; }
    void debug_printByte(uint8_t d) { asm volatile ("" ::: "memory"); }
    void debug_putc(const uint8_t data) { asm volatile ("" ::: "memory"); }

    This results in A, !B. The prints for the value are correct, but the 'IF'
    statement on the next line never triggers. I wish I had an easier way to
    figure this out than simply gaming the memory like this -- I'd rather not
    kick the can down the road with a temporary fix that can't possibly last
    At
    this point, I'll take and try and suggestions I can get!

    So, what would make an "IF" logic test work when DEBUG_puts is defined as
    UART0_puts and fail when DEBUG_puts is defined as asm volatile ("" LL
    "memory"); ?Could it really be timing related after all?

    Thanks for all the help. I'm really learning a lot and trying to do my par

    reading and testing.
     
    lawzaz, Jan 23, 2013
    #7
  8. Did you try my suggestion to look at the generated ARM assembly code to
    see what is really going on ?

    Simon.
     
    Simon Clubley, Jan 23, 2013
    #8
  9. lawzaz

    Tim Wescott Guest

    So show us the if statement that fails.

    Simon's suggestion to look at the assembly is a good one and should be
    followed. Even if you don't think you can understand it you should at
    least try (searching the generated assembly for function names helps a
    lot).

    Don't worry about not being trained in the computer science stuff --
    getting embedded programming right involves getting a whole bunch of
    little details correct that they either never taught you, or that you
    forgot during the post-finals beer bash.

    --
    My liberal friends think I'm a conservative kook.
    My conservative friends think I'm a liberal kook.
    Why am I not happy that they have found common ground?

    Tim Wescott, Communications, Control, Circuits & Software
    http://www.wescottdesign.com
     
    Tim Wescott, Jan 23, 2013
    #9
  10. lawzaz

    lawzaz Guest

    Simon,

    I took a look at the assembly and couldn't figure out where my problem was
    (though I'm trying to learn), but it gave me some insight into some of the
    other problematic behavior. For example, when I set an mfence for
    DEBUG_printf and DEBUG_putc, I didn't realize that that was actuall
    creating
    new functions as the original definitions were aliases anyway. Only
    DEBUG_printByte needed the asm code.

    The failing code is below:

    CAN_Packet is set in an ISR, which is disabled when this code runs. Th
    "IF"
    that fails is
    "if ((CAN_Packet & POWER_SAVE_MASK) < (POWER_SAVE_STOP_COND
    POWER_SAVE_MASK))"

    Interestingly, if I add a print of the value of CAN_Packet and then
    regular
    UART0_puts of a few characters, it works. If I just print the value or i
    I
    just print the characters, it doesn't. Both most be true. It's like th
    memory
    must be called (to prevent optimization?) and it also needs a delay?

    if (CAN_Packet == POWER_SAVE_PACKET) {
    // These must be uncommented for the code to work
    //UART0_printByte(CAN_Packet & 0xFF);
    //UART0_puts("TEXT\n");

    Power_Save_Count++; // Increment counter for consecutive read events
    if ((CAN_Packet & POWER_SAVE_MASK) < (POWER_SAVE_STOP_COND
    POWER_SAVE_MASK)) {
    debug_printf("Stopped.\r\n");
    if (Power_Save_Count == POWER_SAVE_TRIGGER) {
    Modem_upload("SHUTTING_DOWN\r\n",15);
    if (SendFile("ACTIVE") != FR_OK) {
    Modem_upload("ERROR_XFER\r\n",12);
    } else {
    Modem_upload("FILE_SENT\r\n",11);
    }
    Modem_upload_finish();
    Modem_power_toggle();
    }
    else if (Power_Save_Count >= POWER_SAVE_TRIGGER) {
    if (Power_Save_Count >= POWER_SAVE_HEARTBEAT && POWER_SAVE_HEARTBEAT !
    0) {
    Reboot();
    }
    }
    }
    else {
    debug_printf("Moving.\r\n");
    if (Power_Save_Count >= POWER_SAVE_TRIGGER) { Reboot(); }
    else { Power_Save_Count = 0; }
    }
    }
     
    lawzaz, Jan 24, 2013
    #10
  11. lawzaz

    Tauno Voipio Guest


    Is CAN_Packet declared volatile?
     
    Tauno Voipio, Jan 24, 2013
    #11
  12. If CAN_Packet is declared as volatile, your next check is to make sure
    this ISR really _is_ disabled.

    Declare a _volatile_ unsigned long integer and increment it from within
    your ISR _every_ time your ISR is called (ie: make it the first statement
    executed in the ISR).

    Save the current value, but do not print it yet, of this counter just
    before the block of code you posted. Save the value again into another
    variable after the block of code.

    Print both values when it's convenient to do so from within your code.
    If they are not the same, your ISR is not disabled.

    Simon.
     
    Simon Clubley, Jan 24, 2013
    #12
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.