1. This forum section is a read-only archive which contains old newsgroup posts. If you wish to post a query, please do so in one of our main forum sections (here). This way you will get a faster, better response from the members on Motherboard Point.

AT commands: how to detect unsolicited result codes

Discussion in 'Embedded' started by pozzugno, May 8, 2014.

  1. pozzugno

    pozzugno Guest

    As you know, modems (GSM, GSM/GPRS, PSTN, ...) are controlled
    by a host microcontroller through AT commands send to a asyncronous
    serial port. The modem answers with a message.
    Commands and answers are ASCII-based and terminated with <CR>
    and/or <CR><LF>.

    Usually echo is activated for human interaction, but it is usually
    disabled when the modem is controlled by a microcontroller.

    For example (-> micro-to-uC, <- modem-to-uC):
    -> AT<CR>
    <- <CR><LF>OK<CR><LF>
    -> AT+IPR=57600<CR>
    <- <CR><LF>OK<CR><LF>

    Writing a modem driver is simple: push each received character in
    a FIFO buffer inside the ISR. Main task pop the characters in the
    FIFO until the termination (<LF), when a complete answer is found.

    The transmit task is a little harder: a command should be sent
    only after receiving the *complete* answer of the previous command.
    Indeed I tried to send a second commands immediately after the
    first one, but the modem answered and processed only the first.

    I have two problems in this scenario.

    The first is that some answers have termination characters
    <CR><LF> embedded. For example:

    -> ATI4<CR>
    <- SIMCOM_Ltd SIMCOM_SIM900<CR><LF><CR><LF>OK<CR><LF>

    The previous driver is broken for this answer because it would
    detect two answers (the first terminated at the first <LF>).

    I could modify the driver to detect the 4-characters termination
    "OK<CR><LF>", but some answers could end with "ERROR<CR><LF>".
    I could detect both of them as answer termination, but I think
    the modem is free to emit other types of answers with a different
    scheme.

    The second problem is with unsolicited result codes that are
    messages asyncronously transmitted from the modem to the
    host microcontroller.
    They relate to very different functionalities (GSM registration
    status, incoming SMS, incoming voice/data call...) and, most
    of them (but not all!), can be disabled.

    How to modify the driver to understand those unsolicited messages
    too?
    I'm not an expert of modems, but I think (and *hope*) the modem
    doesn't emit an unsolicited message when it has already started
    answering to a previous command send from microcontroller.
    Otherwise the answer and the unsolicited message overlap and would
    be impossible to distinguish them.

    It seems the answers are human-oriented, but it's difficult to
    automate a process to decode them.

    Any suggestions?
     
    pozzugno, May 8, 2014
    #1
    1. Advertisements

  2. pozzugno

    upsidedown Guest

    Just read until timeout. If there are a few character times idle after
    the last character, you could assume that the message has ended,
    especially if the last character was <LF>.

    If autobauding is enabled, there can be a quite long time after the
    "AT" characters echoed and the rest of the response.

    I had a problem with the updated version of the Siemens GPRS modem
    TC35i (but not with the original TC35), which often failed to handle
    ATxyz<CR> sent in a row.

    Finally I had to modify the command transmission to
    1.) Send "A"
    2.) Wait for echo of "A"
    3.) Send "T"
    4.) Wait for echo for "T"
    5.) Send xyz<CR> in a row.

    Which worked every time. The autobauding seemed to slowed considerably
    from the original modem version.
     
    upsidedown, May 8, 2014
    #2
    1. Advertisements

  3. Il giorno giovedì 8 maggio 2014 14:50:24 UTC+2,
    ha scritto:
    It's a nice idea... does it really work?

    The receiving state machine should arm two timeouts: one for the
    no answer (5 seconds? I think it could depends from the actual command
    send) and another one for intra-character time (100ms?).

    The only problem I see with your approach is with unsolicited result
    codes that could be sent randomly from modem. Consider the following:

    -> AT<CR>
    <- <CR><LF>OK<CR><LF>
    <- <CR><LF>RING<CR><LF>

    If the first character of RING unsolicited message is send immediately
    after the last character of OK answer to AT command, your state machine
    could erroneously think it's a single message (and process it in that way).

    This is why I'll never use autobauding with a host microcontroller. Why
    don't you disable it?
     
    giuseppe.modugno, May 8, 2014
    #3
  4. pozzugno

    upsidedown Guest

    I have used this method with MC 68360 and various Power PC with QUICC
    coprocessor for more than a decade. Of course, any controller with a
    free timer with microsecond timer will do.
    Since this is simple 7 bit ASCII, you could insert special character
    like TIMEOUTxxxms into the reception queue and let the high level
    logic make sense of that frame.
     
    upsidedown, May 8, 2014
    #4
  5. pozzugno

    John Speth Guest

    The first is that some answers have termination characters
    Just throw away LFs in the Rx ISR and use CRs for the official
    termination byte.

    JJS
     
    John Speth, May 8, 2014
    #5
  6. pozzugno

    pozzugno Guest

    ha scritto:
    Do you mean pushing into receiving buffer a non-ASCII character (like 0xFF)
    to mark the end of the previous message?
    This could work only if the modem *guarantees* a sufficient delay between
    the last character of a message and the first character of the next message.

    Consider the previous example (AT command from micro, OK answer from modem
    and immediately after RING unsolicited message). What happens if the RING
    event is generated *in the middle* of OK answer transmission? I think the
    modem firmware buffer the RING event and put out the unsolicited message as
    soon as the serial port is free, i.e. immediately after the last character
    of the OK answer. Do the modem firmware wait for some delay before putting
    out the next message?

    In case you're right, how could I know the amount of this delay, considering
    this behaviour isn't documented? It is very difficult to measure it, because
    I need to reproduce the events above... how?

    What is the delay you use to mark the end of a message with your modem?
    I'm using Simcom.
     
    pozzugno, May 9, 2014
    #6
  7. pozzugno

    pozzugno Guest

    Il giorno giovedì 8 maggio 2014 21:12:25 UTC+2, R.Wieser
    ha scritto:
    This is indeed a possible strategy, but it works only if the modem doesn't
    decide to transmit an unsolicited message immediately before the answer of
    the command send by the application (not the driver). Or you're not
    interested in unsolicited message (but I need to decode them).

    Should the program, in every point it uses the modem, decode an answer that
    could be mixed with unsolicited message? I don't think it's a good idea.
     
    pozzugno, May 9, 2014
    #7
  8. pozzugno

    upsidedown Guest

    Or even set the highest bit on the last byte :)
    Typically, the AT command set is just half duplex request/response
    traffic, so framing by idle periods is quite reliable. However, if
    there are full duplex traffic, just read until next idle timeout. Let
    the higher level logic figure out what is response to a request and
    what is spontaneous "RING" messages.

    Of course, the receiver buffer must be able to contain a few of the
    longest messages.
    In most cases 2-5 character time seems to work reliably, without
    spending too long waiting after the whole message. At 9600 bit/s the
    character time is about 1 ms, so an idle timeout of 2-5 ms should be
    OK.

    So even if the delays between characters is longer you expected
    (especially after "AT" if autobauding is used), you get some extra
    idle timeouts in the buffer, but the higher level logic could ignore
    then, if it is within a known response sequence.
     
    upsidedown, May 9, 2014
    #8
  9. ha scritto:
    I'm not considering full-duplex traffic, my example is half-duplex.

    1. Host microcontroller sends completely the command AT<CR><LF>
    2. Modem answers with <CR><LF>OK<CR><LF> (during this msg RING event occurs)
    3. Modem transmits unsolicited message <CR><LF>RING<CR><LF>

    My question was: does the modem waits between steps 2 and 3? If the internal
    firmware of the modem enqueue messages in a global output buffer, it is
    possible to have two consecutive messages without a delay between them.
     
    giuseppe.modugno, May 9, 2014
    #9
  10. pozzugno

    R.Wieser Guest

    Hello Pozzugno,
    You will have to check the docs ofcourse, but I could imagine that such
    unsolicited messages will not occur between the end (and possibly even the
    begin!) of the request you send and the end of the reply for request that it
    sends back.

    As for your described problem ? Check, just before sending the last
    byte/char (the CR/LF) of your request if something has come in. If so,
    delay sending that last char until you have parsed/purged the received data.
    You're quite correct in that, which is the reason I think that the writers
    of that embedded modem software will have thought of it to and (quite
    likely) will hold off any such unsolicited message until a running request
    has been completeley finished. Again, I would suggest you check the
    (interfacing) specs coming with(?) that modem for it.

    One other thing: does that modem have any hardware control (RTS/CTS) ? In
    that case I could imagine that a CTS stays false for as long as a command is
    executing and data is send your way (assuming half-duplex here). If an
    unsolicited message is than "tacked on" the reply you should see the CTS
    going true in between.

    Regards,
    Rudy Wieser


    -- Origional message:
    <> schreef in berichtnieuws
    ...
    Il giorno giovedì 8 maggio 2014 21:12:25 UTC+2, R.Wieser
    ha scritto:
    This is indeed a possible strategy, but it works only if the modem doesn't
    decide to transmit an unsolicited message immediately before the answer of
    the command send by the application (not the driver). Or you're not
    interested in unsolicited message (but I need to decode them).

    Should the program, in every point it uses the modem, decode an answer that
    could be mixed with unsolicited message? I don't think it's a good idea.
     
    R.Wieser, May 9, 2014
    #10
  11. pozzugno

    pozzugno Guest

    R.Wieser ha scritto:
    Indeed your imagination is wrong :)
    I think AT commands are oriented to human interaction and they surely can be
    send by typing the characters on a keyboard (you have the possibility to
    correct what you have typed by pressing backspace). This means that the
    command you are sending can take a long time to finish. In the meantime,
    the modem can send you an unsolicited message. I already test this situation.

    It couldn't work. When I decide to send last char (even after a long delay),
    at the same time the modem could decide to send an unsolicited message.

    Unfortunately you're wrong :-(
    Why? RTS/CTS are used for flow-control: the receiving could say to the
    transmitter to stop transmitting if the incoming queue is almost full.
    I don't think RTS and CTS signals move with the transmitting of requests,
    answers or unsolicited messages.
     
    pozzugno, May 9, 2014
    #11
  12. To be honest, I'm not really seeing what the problem is.

    I would have implemented this as a state machine in the upper layers of
    the code with clearly defined state transitions depending on (1) timeouts,
    (2) the responses you are waiting for and (3) unsolicited events such
    as ring.

    The next layer down would include a parser which extracts any printable
    text, on a per line basis, from the modem and either tokenises it or
    passes it as-is back up to the state machine.

    If the state machine sees a ring while waiting for OK, it can decide
    if it wants to transition to a new state at that point or wait for
    the expected response (or timeout).

    That means in the event of RING followed on the next line by OK, your
    state machine would see two distinct responses from the modem.

    You would of course have to make sure your state machine always
    terminated any possible successful connect so you were not running up
    connect charges after recovery from any unexpected events.

    Is the above how others here would implement this or would you
    use a different approach ?

    Simon.
     
    Simon Clubley, May 9, 2014
    #12
  13. pozzugno

    R.Wieser Guest

    Hello Pozzugno,
    Well, you're right about that humans could take a long time to finish typing
    a command, so the "from the begin" idea is out. But I still do not think
    that there will be much chance (if any) that an unsolicited message will be
    send while another command is being executed (for multiple reasons).
    You're right, stated like that it seems like an unsolvable "race condition"
    problem. Nevertheless, it must have been solved time-and-again, otherwise
    simple old-style modem communication between computers would never have
    passed its beta stage. :)

    Is it possible that there are few different unsolicited messages and by
    their fixed nature are fairly easy to recognise/discern from an expected
    result, and by that feat rather easily removed when its recieved before an
    expected reply ?
    I don't think so either. I rather thought it could be a by-product of the
    modem being busy internally (executing the command you just send, or simple
    being busy sending you data), and therefore not able to receive any data
    itself.

    Well, I think I'm at the end of my ideas. Sorry I could not help you.

    Regards,
    Rudy Wieser


    -- Origional message:
     
    R.Wieser, May 9, 2014
    #13
  14. pozzugno

    upsidedown Guest

    If the modem sends "RING" at the same time time as you start sending
    I have no idea how each modem on the market behaves.
     
    upsidedown, May 9, 2014
    #14
  15. pozzugno

    pozz Guest

    Il 09/05/2014 19:21, ha scritto:
    Yes, you're right. But my example above aren't full-duplex. AT command
    is *completely* send out (step 1 is finished) when the modem starts
    answering (step 2). And the modem *completely* answered with OK (step 2
    is finished) when it starts emitting unsolicited RING message (step 3).

    This is definitely a half-duplex communication.

    If the modem doesn't *guarantee* a minimum delay (4-5 characters time)
    between two output messages, are you agree your method (separating
    frames thanks to a delay) doesn't work?
     
    pozz, May 10, 2014
    #15
  16. pozzugno

    pozz Guest

    Il 09/05/2014 15:03, R.Wieser ha scritto:
    Why not? I'm thinking how the embedded GSM modem firmware could work.
    There are many threads running that can push output messages in a output
    queue.

    One thread could manage the GSM network, so it receives signals that
    incoming call is in progress. This thread could push "RING" message in
    the output buffer queue.

    Another thread could manage the AT requests on the serial port and could
    push "OK" answers.

    At a certain time, the output queue could contain two messages, emitted
    one after the other in a transmitting UART ISR.

    I agree with you and this is why I asked for a solution of this problem.
    I don't think the problem hasn't a solution, I'm quite sure I'm using a
    wrong method.

    Indeed it seems the only possible solution, even if it isn't simple to
    implement. The receiving driver should detect a certain number of
    unsolicited messages and filter them out to the application layer that
    wants only the answers to commands. There many unsolicited messages:

    <CR><LF>RING<CR><LF>
    <CR><LF>CONNECT ...<CR><LF>
    <CR><LF>BUSY<CR><LF>
    <CR><LF>DIALTONE<CR><LF>
    <CR><LF>+CGMS...<CR><LF> (or something similar)

    I have the possibility to implement *all* the possible unsolicited
    messages, but I don't think it is feasible because there are many and
    some of them couldln't be documented.

    Alternatively I can capture and filter only the unsolicited messages I
    want to decode and analyze. The other unsolicited messages will not be
    recognized as such so they must be passed to the application layer.
    The application should be ready to receive not only answers to commands
    it send, but also other messages that should be discarded because not
    useful unsolicited messages.
     
    pozz, May 10, 2014
    #16
  17. pozzugno

    pozz Guest

    Il 09/05/2014 14:29, Simon Clubley ha scritto:
    Even the "per line basis" parser approach doesn't work everytime.
    The answer to the AT command that sends SMS isn't <CR><LF> as usual, but
    it is the 2-characters prompt "> ". The lower layer (the driver?) can't
    decide to wait for <CR><LF> before passing the incoming characters to
    the application layer.

    One possibility is to change at run-time the ending suffix: normally it
    is <CR><LF>, but it could be "> ". Anyway this suffix will be used only
    for answers to pass to the application layer. At the same time
    unsolicited messages (ending with <CR><LF>) could arrive and must be
    managed.

    You're thinking of a lower level that is able to separate two messages,
    even if they are strictly *consecutive*: answer-unsolicited or
    unsolicited-answer.

    This is the starting question: what is the best method to separate
    incoming characters from modem into *distinct* messages in *all*
    possible situations (answers, unsolicited...)?
    <CR><LF> prefix/suffix could be a method, but in some situation the
    suffix changes.

    Are you thinking of a lower level that pass to the application layer
    *all* the messages: answers and unsolicited? IMHO it's not feasible. All
    the upper complex FSMs (there are many, one for sensing SMS, another for
    receiving SMS, another for making voice calls, another for making data
    calls, ...) should decide, in *every* state, what to do in case of
    unsolicited messages.
     
    pozz, May 10, 2014
    #17
  18. pozzugno

    upsidedown Guest

    In the interrupt service routine (which should be kept very simple to
    avoid blocking other interrupts) just use the idle timeout.

    It is to the upper level logic to find out the actual framing, e.g. by
    knowing the structure of the expected response (such as ending in
    <LF>).

    If it can't make any sense, just resend the command after a while
    (100-1000 ms). This works well in practice.
     
    upsidedown, May 10, 2014
    #18
  19. pozzugno

    upsidedown Guest

    Why bother with a fuss like that above. If your application layer
    doesn't understand the response, just resend the original command a
    few times until you get a valid response or declare the connection
    dead.

    For the resend interval use something odd like 3.13 or 2.71 seconds,
    since if there are similar timers at the end, they would typically
    expire after 1, 2 or 3 seconds :).
     
    upsidedown, May 10, 2014
    #19
  20. In that case, my first try at implementing this would be to switch to
    traditional per character parsing and tokenising the response. If you
    see "> ", then that's one token and it can be pushed upstairs without
    any wait.

    If OTOH you see a alphanumeric character, you continue scanning until
    you see a non-alphanumeric character, (although you may choose to treat
    a space as part of the sequence and look for a non-printable character
    instead).

    The FSM should know what are the possible responses in each state, both
    expected and unsolicited. I agree it can get complicated, but if you
    are getting those messages you have to handle them somehow, even if
    you transition back to the same state, thereby ignoring the message.
    It matters because you need to handle error recovery in a robust manner.
    Your commands may have side effects, such as changing a configuration,
    sending a message, or establishing a connection.

    You can't just blindly send them multiple times. You might get away with
    that in some circumstances, but not always.

    There's enough lousy embedded software out there without us adding to
    the problem. :)

    Simon.
     
    Simon Clubley, May 12, 2014
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.