IRQ11 shared by a number of devices on a ThinkPad notebook (IO-APIC disabled?)

Discussion in 'IBM' started by Frantisek.Rysanek, May 4, 2007.

  1. Dear everyone,

    where I work, a colleague of mine's got a ThinkPad R51 1829-R6G. A
    while ago he's paid a visit, complaining about the sluggishness of his
    notebook, and that some PC Cards didn't work.

    I've discovered that maybe 10 or 12 devices shared IRQ 11. Never seen
    this before, certainly not on an ACPI-capable system with an IO-APIC
    on board. This degree of IRQ sharing inherently means awful CPU load
    consisting of calls to interrupt service routines.

    Fumbled through the BIOS setup, found all PCI INT lines configured for
    IRQ 11 (eight of them). Toggled them all to "auto-select", but still
    the BIOS would assign IRQ 11 to all PCI devices.
    Flashed the BIOS to the most recent version available (1.35, I
    believe), to no avail - still the same, BIOS defaults mean "set
    everything to IRQ 11", auto-select does just the same, there's no way
    to select specific APIC IRQ's above IRQ 15.

    Is this possibly a BIOS bug of some sort? The machine's got an ICH4-M
    south bridge, this one has an IO-APIC with 24 IRQ lines. IRQ's 16
    through 23 should be available!
    Is the IO-APIC disabled? If yes, how do I re-enable it? I couldn't
    find a relevant entry in the BIOS setup.

    Found this out in Windows XP, verified using a Linux live CD - it's
    definitely a BIOS issue.

    Not a word about this in the online tech support information. I've
    seen ACPI BIOS / IRQ routing issues before, mostly on no-name PC
    hardware.

    IBM/Lenovo did respond swiftly to my tech support request and were
    definitely very kind in dealing with me, and I'm grateful for that
    sort of treatment :) Yet in the end they haven't quite acknowledged a
    mere existence of the issue. A country-wide authorized service center
    (external partner company) didn't help, either. That's why I'm
    resorting to asking this in a public newsgroup, perhaps someone out
    there has more experience, perhaps it's a problem on my part :)
    The IBM/Lenovo notebooks are otherwise an incredibly robust hardware
    platform.

    Any ideas are welcome

    Frank Rysanek
     
    Frantisek.Rysanek, May 4, 2007
    #1
    1. Advertisements

  2. Frantisek.Rysanek

    Arno Wagner Guest

    Actually no. It means that each time IRQ 11 is called an awful lot of
    devices have to be polled to find out whether they were the source.
    Polling can take really long.
    No idea what the issue actually is. But the sliggishness is indeed
    likely caused by it. Can you manually addisgn different IRQs?

    Arno
     
    Arno Wagner, May 4, 2007
    #2
    1. Advertisements

  3. Dear Arno,

    thanks for responding in the first place :)
    Right. The number of ISR calls corresponding to a shared IRQ line
    grows with a square of the number of devices on that shared IRQ line.
    (That is, unless the ISR's return a trustworthy result code, and the
    growth is only somewhat less if they do - this is OS-dependent.)
    If those same devices each had their very own IRQ line, the growth
    would be linear.

    The PC card that ultimately got this particular system to its knees
    was a Gb Ethernet adaptor with a Cardbus attachment to the host :)
    The IBM BIOS setup actually does contain eight relevant entries in the
    "PCI/PnP configuration" screen (or whatever it's called). The original
    intention was probably nice, but practically those configurable
    entries are completely useless.

    The IO-APIC included on-chip in the ICH4-M south bridge used in the
    Thinkpad has 8 dedicated PCI interrupt inputs (apart from the standard
    XT/AT-PIC IRQ0 through 15). Even in the package pinout, the ICH4-M
    chip's got the standard set of PCI IRQ inputs called INTA through INTD
    (for a single PCI bus segment), along with four more optional inputs
    called INTE throught INTH or something like that (GPIO pins with an
    alternative function). In an OS that can use the APIC, these eight
    input pins are normally routed to IRQ numbers from 16 to 23, or some
    other arbitrary high numbers (OS-dependent behavior), so that the
    legacy IRQ's 0 through 15 remain available to legacy ISA devices
    (internally present in the chipset of every IBM-compatible PC to date)
    without IRQ sharing.
    The 8 additional PCI interrupt inputs of the IO-APIC can be used for
    both ICH on-chip and other off-chip PCI devices. On-chip and off-chip
    PCI devices can share individual PCI interrupt lines.

    The IBM BIOS clearly contains configuration entries for these 8 PCI
    interrupt inputs, and the labels on those entries suggest that they
    were intended to allow you to "hard-wire" a particular INT input to a
    particular IRQ number. (Silly as it seeems, in an APIC-based system,
    where the IRQ numbers as seen by the software running on the host CPU
    are more or less arbitrary.)
    In other words, I was quite curious when I first saw them - I was
    wondering if these would offer the legacy IRQ lines (0 through 15) or
    some expected APIC based IRQ numbers that the OS was expected to come
    up with, or what the hell.
    It has turned out that the BIOS default is to set all these entries as
    "hard-wired to IRQ11". Found out by reverting to BIOS defaults on
    purpose, to see what happens. (Originally I thought this setup was the
    result of someone's past human configuration attempts.) Anyway so I
    tried changing the hard-wired IRQ11 to something different, and the
    only other option is "auto-select". Which practically settles on IRQ11
    anyway :-(

    As for trying to change this in the operating systems, I believe that
    is pointless.
    Both Linux and modern Windows (such as XP) deliberately prevent you
    from fiddling with IRQ numbers on PCI devices. This makes perfect
    sense - the answer to "how do I change the IRQ" is that "you don't,
    the BIOS + OS + PCI bus do that for you." A human administrator
    doesn't have a chance to cope with the complexities of APIC-based IRQ
    routing in the system hardware, which has chipset-specific basis and
    board-specific custom configurations. The only correct way to
    implement this is "automagically", via a set of standards-based HW and
    SW interfaces such as APIC's, ACPI or the older MPS. Consequently, the
    IRQ numbers become largely arbitrary - just an allocated slot in the
    CPU's local APIC, wired by the BIOS+OS-driven black magic to some IO-
    APIC input, to an actual PCI device...

    Windows XP does run with an ACPI HAL on the culprit machine, and I'm
    happy with that. There's nothing else you can do (ACPI can't be turned
    off in the ThinkPad BIOS, so I couldn't even try to reinstall Windows
    with MPS-based HAL).

    Linux does offer a few bootloader options (kernel command line
    options) to modify the methods of IRQ routing setup: pci=routeirq,
    pci=noacpi, acpi=off. None of this has any effect, even in Linux all
    the PCI devices share IRQ 11.

    I believe it's up to the BIOS to get the ACPI/MPS tables right, or
    maybe to enable the IO-APIC in the first place.
    I was just wondering if someone knew about a BIOS update, or some IBM
    configuration utility, that might be able to achieve that - or perhaps
    if I'm just missing something obvious...

    Thanks for everybody's attention, have a nice weekend :)

    Frank Rysanek
     
    Frantisek.Rysanek, May 4, 2007
    #3
  4. Frantisek.Rysanek

    don't look Guest

    I've seen it mentioned that turning off PnP in the bios then assinging IRQ's
    manually could work.
     
    don't look, May 5, 2007
    #4
  5. Frantisek.Rysanek

    Franc Zabkar Guest

    FYI ...

    The Importance of Implementing APIC-Based Interrupt Subsystems on
    Uniprocessor PCs:

    http://www.microsoft.com/whdc/system/sysperf/apic.mspx

    "In 1996, Microsoft was told by Intel to expect all its machines to
    have I/O APICs by 1998. So, the IRQ arbiter was designed to cause
    every PCI device to share its IRQ with ACPI, when ACPI was enabled.
    This solved most of the interrupt constraint problems, but caused a
    huge amount of IRQ sharing."

    "On the machine being used to write this paper, there are 11 devices
    sharing IRQ 11. If any one of them interrupts, the operating system
    begins at the head of the ISR chain and examines each device until it
    finds the one that interrupted. Each hardware access can take several
    microseconds, and each ISR may access its device several (or even
    many) times. The result is that hardware accesses cause delays in the
    processing of all other tasks on the machine. Each interrupt can
    potentially cause a delay of most of a millisecond during which no
    other work can be done. This makes it very difficult to guarantee that
    time-sensitive devices, such as audio devices, function correctly.
    (This machine actually glitches its audio whenever there is a large
    amount of other activity in the machine.)"

    - Franc Zabkar
     
    Franc Zabkar, May 5, 2007
    #5
  6. I've seen it mentioned that turning off PnP in the bios then assinging IRQ's
    I'm afraid that advice is very generic, and doesn't seem to apply to
    the Thinkpad. There's no way to disable PnP support in the BIOS, and
    assigning IRQ's manually likely wouldn't work for the APIC-based
    IRQ's, even if it was possible.

    I am aware that specifically Award BIOSes typically allow you to
    select, for each IRQ (of a selected set where this is supported), one
    of the following two options:
    A) "reserved" (for legacy ISA devices) - route this IRQ to the
    corresponding physical ISA IRQ line only
    B) "PCI/ISA PnP" - means leave assignment/usage of this IRQ number up
    to the PnP subsystem in the BIOS. The PnP BIOS will likely route the
    IRQ to one of the on-chip/on-board devices, either ISA/LPC-based or
    PCI-based. Or if you still happen to have an ISAPnP compatible ISA
    addon board, that will be involved, too.

    I believe this generic Award functionality is what you are referring
    to...

    Frank Rysanek
     
    Frantisek.Rysanek, May 8, 2007
    #6
  7. Just a few more comments: this is the latest BIOS for the machine,
    i.e. 3.21.
    If I boot Linux:
    - it does detect and use ACPI, including for interrupt routing
    - it says local APIC detected but disabled by BIOS.
    Tried re-enabling it, to no effect.
    - it says "no IO APIC detected".
    - it says "no SMP motherboard detected", i.e. no MPS, no MPtable

    As if the machine indeed had no APIC at all. Yet the ICH4-M does
    contain one.
    Without ACPI support and without a valid MPtable, it's no use anyway,
    even if I could enable it in the ICH4.

    Frank Rysanek
     
    Frantisek.Rysanek, May 11, 2007
    #7
  8. Frantisek.Rysanek

    joeuser

    Joined:
    Dec 3, 2009
    Messages:
    12
    Likes Received:
    0
    Location:
    Everywhere.
    I found this thread after the Ethernet port on a customer's Lenovo Thinkpad R61 running WinXP Professional stopped working. I enabled the "Intel Virtualization Technology" option in the bios because I was just curious to see how it would affect the performance of laptop. To my surprise the Ethernet port stopped working and checking in the Device Manager of Windows showed the yellow exclamation mark on the device and stated the device couldn't start. Even when I changed back the settings in the bios and then updated the bios the Ethernet port still didn't work. I finally realized that the change that I made in the bios had bumped the Ethernet controller off the IRQ it had. So to get back the IRQ the Ethernet controller needed I disabled all the IRQs in the bios then booted into Windows. After POST an error message came up stating that no IRQs were assigned. I just hit Esc key to skip this and continued the boot process. This forced Windows to assign the IRQs for all the devices instead of the Bios. And presto the Ethernet port was back working. I rebooted the laptop and went back into the bios to change the IRQ settings to "Auto Select" because the error message would continue to appear during the boot process if I hadn't. Before the first time that I changed the IRQ settings they were all set to 11 like stated in the above posts. Hope this helps someone in the future.
     
    joeuser, Sep 21, 2013
    #8
  9. Frantisek.Rysanek

    lanzyga

    Joined:
    Feb 11, 2014
    Messages:
    1
    Likes Received:
    0
    Location:
    St. Paul, MN
    Spreading Thinkpad PCI IRQ mappings

    Through trial and error - and insight into how IBM architects "THINK" (I was an IBMer for 20+ years) - I found out how to get the older Thinkpads to spread out PCI bus interrupt mappings to more than just IRQ 11. (This effort was done while getting ESXi to run on some surplus laptops at my employer so that I could deploy a "small model" virtual appliance for some of our small sites. Laptops are being used because they have a built-in UPS commonly called a "battery" :) )

    FYI, this process was done on a Thinkpad T400, but I suspect it applies to all Thinkpads which have a Config -> PCI interface. (My new T430s Thinkpad does not - I haven't yet looked to see how it maps PCI interrupts... since it has a more modern chipset, I would hope it allows operating systems to re-assign interrupts more easily.)

    The "trick" is that "Auto Select" really should be called something like more like "Default". What Auto Select really does is:
    1. Determine the list of "available" low IRQs for assignment. (IRQ11 is always in this list.)
    2. For each listed PCI IRQ mapping (INTA-INTH on my T400):
      • If there is no previous assignment, assign a low IRQ "round robin" from the list in step 1.
      • If there is a previous assignment but that IRQ is no longer in the list, assign a new IRQ "round robin" from the list. (An example of this case would be if the vector was previously mapped to IRQ4 with the motherboard/docking station serial port disabled, then the serial port is enabled for COM 1 (which uses IRQ 4).)
      • Otherwise (default case): assign the same IRQ as was used previously.

    But the problem is - IBM/Lenovo shipped these with all of the PCI interrupts mapped to IRQ 11 - how do we get out of that as the "default" IRQ? One answer (which worked for me) is:
    • Go into BIOS and set ALL of the PCI IRQ mappings to Disabled.
    • F10 to exist and reboot. The reboot will fail miserably - because there is no boot device available (the hard drive and CD ROM aren't available).
    • Power-cycle, go back into BIOS, set all of the PCI IRQs to Auto Select. F10 to save/reboot.
    • Presto! The old IRQ11 mappings are gone... the PCI IRQs will now be nicely "spread" across all of the available IRQs (including, but not exclusively, IRQ 11).

    I confirmed this behavior with both Linux and ESXi (ESXi does a really nice job showing all of the PCI device/interrupt mappings in the vmkernel.log file). As noted previously, this would likely help out significantly with Windows performance on these devices.

    Probably somewhat late to be of a lot of use - but thought I'd share it anyways.

    Garrett
     
    Last edited: Feb 11, 2014
    lanzyga, Feb 11, 2014
    #9
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.