Signaling Events from Interrupt Handlers

Best way to wake multiple threads from interrupt?

I want to make a character device driver that passes the same data to all tasks that are reading it. It is not so important whether the data is queued or if just latest sample is retrieved. Problem is just how to wake up the waiting threads.

At the most primitive level, a thread can be waiting for a semaphore, a signal, or a message queue (not empty or not full). Then there are higher level level wrappers around these like mutexes, semaphores, poll waits, etc. But under the hood those are the three fundamental wait mechanisms. Any could be used to accomplish what you want.

In NuttX, some additional effort was put into the design of the signalling side of each of the IPCs so that they could be easily used by interrupts handlers. This behavior is unique to NuttX; POSIX says nothing about interrupt handlers. As a result, we will be talking about primarily non-portable OS interfaces.

So far I've considered the following options:

And you basically have gone through the list of wait mechanisms:

Message Queues

1) Open a message queue when the device is opened (a new queue for each task) and keep them in a list. Post to a non-blocking endpoint of these queues in the ISR. Read from a blocking endpoint in the device read(). I would need to generate names for the message queues, as there doesn't seem to be anonymous message queues?

When you start a project. It is a good idea to decide upon a common IPC mechanism to base your design on. POSIX message queues are one good choice to do that: Assign each thread a message queue and the main() of each thread simply waits on the message queue. It is a good architecture and used frequently.

However, I would probably avoid creating a lot of message queues just to support the interrupt level signaling. There are other ways to do that that do not use so much memory. So, if you have message queues, use them. If not, keep it simple.

Semaphores

2) Allocate a semaphore per each device open and keep them in a list. Post the semaphores when new data is available in a shared buffer. Read the data inside sched_lock().

If you don't have an architecture that uses message queues, and all of these threads are waiting only for the interrupt event and nothing else, then semaphores would work fine too. You are basically using semaphores as condition variables in this case so you do have to be careful: See http://tech.groups.yahoo.com/group/nuttx/message/1860

NOTE: You do not need multiple semaphores. You can do this with a single semaphore. If the semaphore is used for this purpose then you initialize it to zero: sem_init(&sem, 0, 0). Since the semaphore is initialized to zero, each time that a thread joins the group of waiting threads, the count is decremented. So a simple loop like this would wake up all waiting threads:

int svalue;
int ret;

for (;;)
  {
    ret = sem_getvalue(&sem, &svalue);
    if (svalue < 0)
      {
        sem_post(&sem);
      }
    else
      {
        break;
      }
    }

NOTE: This use of sem_getvalue() is not portable. In many environments, sem_getvalue() will not return negative values if there are waiters on the semaphore.

The above code snippet is essentially what the NuttX pthread_cond_broadcast() does (see nuttx/sched/pthread_condbroadcast.c). In NuttX condition variables are really just wrappers around semaphores that give them a few new properties. You could even call pthread_cond_broadcast() from an interrupt handler: See http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_cond_signal.html for usage information.

Neither of the above mechanisms are portable uses of these interfaces. However, there is no portable interface for communicating directly with interrupt handlers.

NOTE: There is possibility of improper interactions between the semaphore when I is used for signaling and priority inheritance. In this case, you should disable priority inheritance on the signaling semaphore using sem_setprotocol(SEM_PRIO_NONE). See the signaling semaphore Wiki Page for further information.

Signals

3) Store the thread id's in a list when read() is called. Wake up the threads using sigqueue(). Read the data from a shared buffer inside sched_lock().

Signals would work fine too. Signals have a side-effect that is sometimes helpful and sometimes a pain in the butt: They cause almost all kinds of waits (read(), sem_wait(), etc.) to wake up and return an error with errno=EINTR.

That is sometimes helpful because you can wake up a recv() or a read() etc., detect the event that generated the signal, and do something about it. It is sometimes a pain because you have to remember to handle the EINTR return value even when you don't care about it.

The POSIX signal definition includes some support that would make this easier for you. This support is not currently implemented in NuttX. The kill() interface for example (http://pubs.opengroup.org/onlinepubs/009695399/functions/kill.html) supports this behavior:

“If pid is 0, sig will be sent to all processes (excluding an unspecified set of system processes) whose process group ID is equal to the process group ID of the sender, and for which the process has permission to send a signal.

“If pid is -1, sig will be sent to all processes (excluding an unspecified set of system processes) for which the process has permission to send that signal.

“If pid is negative, but not -1, sig will be sent to all processes (excluding an unspecified set of system processes) whose process group ID is equal to the absolute value of pid, and for which the process has permission to send a signal.”

NuttX does not currently support process groups. But that might be a good RTOS extension. If you and others think that would be useful I could probably add the basics of such a feature in a day or so.

poll()

Is there some better way that I haven't discovered?

The obvious thing that you did not mention is poll(). See http://pubs.opengroup.org/onlinepubs/009695399/functions/poll.html . Since you are writing a device driver, support the poll() method in your driver seems to be the natural solution. See the drivers/ directory for many examples, drivers/pipes/pipe_common.c for one. Each thread could simply wait on poll(); when the event occurs the driver could then wake up the set of waiters. Under the hood, this is again just a set of sem_post's. But it is also a very standard mechanism.

In your case, the semantics of poll() might have to be bent just a little. You might have the bend the meaning of some of the event flags since they are all focused on data I/O events.

Another creative use of poll() for use in cases like this: See http://tech.groups.yahoo.com/group/nuttx/message/2579:

> That would be something great! PX4 project has that implemented somehow
> (in C++), so maybe - if license permits - it could be ported to NuttX in
> no time?

https://pixhawk.ethz.ch/px4/dev/shared_object_communication

I don't know a lot about this, but it might be worth looking into if it matches your need.