| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Similarly to message sends, message receives in GM are regulated by a simple token-passing mechanism: Before a message can be received, the client software must provide GM a receive token that allows the message to be received and specifies a buffer to hold the received data.
After initialization, the client implicitly possesses all
gm_num_receive_tokens() receive tokens. The client software
grants receive tokens to GM by calling
gm_provide_receive_buffer(port, buffer, size,
priority), indicating that GM may receive any message into
buffer as long as the size and priority fields of
the received message exactly match the size and priority
fields passed to gm_provide_receive_buffer(). Eventually, GM
will use the buffer indicated by message and size to receive
a message of the indicated size and priority. Unlike some
messaging systems, GM requires that the size of the received
message match the token size exactly. GM will not use the next
larger sized receive buffer when a receive buffer of the correct size is
not available. All receive buffers passed to
gm_provide_receive_buffer must DMAable. They must also be
aligned or be within memory allocated using gm_dma_*alloc() to
ensure that messages can be DMAed into the buffer, and must be at least
gm_max_length_for_size(size) bytes long.
Typical GM clients will provide at least 2 receive buffers for each size
and priority of message that might be received to maximize performance by
allowing one buffer to be processed and replaced while the network is
filling the other. However, 1 receive buffer for each size-priority
combination is sufficient for correct operation. Additionally, it is
almost always a good idea to provide additional buffers for the smallest
sizes, so that many small messages may be received while the host is busy
computing. There is no need to provide tokens for receives smaller than
gm_min_message_size().
After providing receive tokens, code may poll for pending events using
gm_receive_pending(port), which returns a nonzero value if a
receive is pending or zero if no event has been received.
gm_next_event_peek (struct gm_port *p, gm_u16_t *sender)
can also be used to peek at the event at the head of the queue.
The return value is the event type (zero if no event is pending).
The sender parameter will be filled with the sender of
the message if the event is a message receive event.
The client
may also poll for receives using gm_receive(port), which
returns a pointer to a event structure of type gm_event_t.
If no recv event is in the receive queue, a pointer to a fake receive
event of GM_NO_RECV_EVENT will be returned. The event returned
by gm_receive() is only guaranteed to be valid until the
next call to gm_receive().
There are several variants of gm_receive() available, all of
which can safely be used in the same program.
gm_receive()
GM_NO_RECV_EVENT if
none is pending.
gm_blocking_receive()
gm_blocking_receive_no_spin()
Once the client has obtained a receive event from a
gm_*receive*() function, the client should either process the
event if the client recognizes the event, or pass the event to
gm_unknown() if the event is unrecognized. All fields in the
receive event are in network byte order, and must be converted to
host byte order as specified in section See section 10. Endian Conversion.
The client is not required to handle any receive events, and may simply
pass all events to gm_unknown(), but any useful GM program will
handle GM_RECV_EVENTs or GM_HIGH_RECV_EVENTs in order to
access the received data. The receive event types that the client
software may choose to recognize are as follows (GM internal events are
not listed):
GM_ALARM_EVENT
GM_ALARM_EVENTs should be treated as an unknown event and passed
to gm_unknown(). However, because client alarm handlers are
called within gm_unknown() when gm_unknown() receives a
GM_ALARM_EVENT, it can be useful for a program to perform alarm
polling only after passing GM_ALARM_EVENTs to
gm_unknown(), as in the `test/gm_allsize.c' example program.
See the documentation for gm_set_alarm() for more information.
GM_RECV_EVENT
GM_HIGH_RECV_EVENT
event->recv structure.
length
size
buffer
gm_provide_receive_buffer(), which allowed this receive to occur
sender_node_id
sender_port_id
tag
gm_provide_receive_buffer_with_tag() or
0 if gm_provide_receive_buffer() was used instead
type
GM_HIGH_RECV_EVENT indicates the receipt of a high-priority
packet. GM_RECV_EVENT indicates the receipt of a low-priority
packet.
GM_PEER_RECV_EVENT
GM_HIGH_PEER_RECV_EVENT
These events may be safely ignored (passed to gm_unknown()), in
which case the event will be converted to a normal GM_RECV_EVENT
and passed to the client in the next call to a gm_*receive*()
function.
These events are just like the normal GM_RECV_EVENT and
GM_HIGH_RECV_EVENT events, but indicate that the sender port id
is the same as the receiver port id. Most GM programs should handle
these events directly just like they handle normal receive events.
length
size
buffer
gm_provide_receive_buffer(), which allowed this receive to occur
sender_node_id
sender_port_id
tag
gm_provide_receive_buffer_with_tag() or
0 if gm_provide_receive_buffer() was used instead.
type
PEER event types indicate that the sender port number is the
same as the port number. The HIGH event types indicate that the
message was sent with high priority.
GM_FAST_RECV_EVENT
GM_FAST_HIGH_RECV_EVENT
GM_FAST_PEER_RECV_EVENT
GM_FAST_HIGH_PEER_RECV_EVENT
These events may be safely ignored (passed to gm_unknown()), in
which case the event will be converted to a normal GM_RECV_EVENT
and passed to the client in the next call to a gm_*receive*()
function. The conversion process will copy the receive message
from the receive queue into the receive buffer.
These types indicate that a small-message receive occurred with the small
message stored in the receive queue for improved small-message
performance. The PEER event types indicate that the sender port
number is the same as the port number. The HIGH event types
indicate that the message was sent with high priority.
If your program uses any small messages that are immediately processed
and discarded upon receipt, then your program can improve performance by
processing these messages directly. If after examining the message your
program determines that it needs the data copied into the buffer, it can
either call gm_memorize_message() to do so or can pass the event
to gm_unknown().
message
gm_receive()
length
size
buffer
gm_provide_receive_buffer(), which allowed this receive to occur
sender_node_id
sender_port_id
tag
gm_provide_receive_buffer_with_tag() or
0 if gm_provide_receive_buffer() was used instead.
type
PEER types indicate that the sender port number is the same
as the port number. The HIGH types indicate that the message was
sent with high priority.
Note that although the receive data is in the receive queue and no
receive buffer was used to store the received message, the client must
have provided an appropriate receive buffer before the receive could
take place, and this buffer is passed back to the client in the fast
receive event. If the client needs to store the data *message
past the next call to gm_receive(), then the client should copy
*message into *buffer using gm_memorize_message(),
which is simply a version of bcopy() optimized for copying
aligned messages. After calling gm_memorize_message(), the fast
receive event becomes equivalent to a normal receive event.
GM_NO_EVENT
GM_RAW_RECV_EVENT
event->recv structure:
length
buffer
GM_SENT_EVENT
This type indicates that one or more sends completed.
Developers using the GM-1.1 API should never see this event
type, as it is generated only if the client calls the GM-1.0
gm_send() function, which is deprecated in favor of the superior
gm_send_with_callback() functions.
event->sent.message_list points to a null-terminated array of
void pointers, which are message pointers from earlier
gm_send() calls that have completed successfully. For each pointer
in this array, a send token is implicitly returned to the client.
Although the number of receive events may seem daunting at first glance, almost all of the event types can be ignored. The following receive dispatch loop is fully functional for a nontrivial application that accepts messages ports, accepts only small control messages sent with high priority, and accepts low priority messages of any size:
{
struct gm_port *my_port;
gm_recv_event_t *e;
void *some_buffer;
...
while (1) {
e = gm_receive (my_port);
switch (gm_htohc (e->recv.type))
{
case GM_HIGH_RECV_EVENT:
/* Handle high-priority control messages here in bounded time */
gm_provide_recv_buffer (my_port,
gm_ntohp (e->recv.buffer),
gm_ntohc (e->recv.size),
GM_HIGH_PRIORITY);
break;
case GM_RECV_EVENT:
/* Handle data messages here in bounded time */
gm_provide_recv_buffer (my_port, some_buffer,
gm_ntohc (e->recv.size),
GM_LOW_PRIORITY);
break;
case GM_NO_RECV_EVENT:
/* Do bounded-time processing here, if desired. */
break;
default:
gm_unknown (my_port, e);
}
}
}
|
However, the following implementation is slightly faster because it handles control messages without copying them into the receive buffer:
{
struct gm_port *my_port;
gm_recv_event_t *e;
void *some_buffer;
...
while (1) {
e = gm_receive (my_port);
switch (gm_ntohc (e->recv.type))
{
case GM_FAST_HIGH_PEER_RECV_EVENT:
case GM_FAST_HIGH_RECV_EVENT:
/* Handle high-priority control messages here in bounded time */
gm_provide_recv_buffer (my_port,
gm_ntohp (e->recv.buffer),
gm_ntohc (e->recv.size),
GM_HIGH_PRIORITY);
break;
case GM_FAST_PEER_RECV_EVENT:
case GM_FAST_RECV_EVENT:
gm_memorize_message (gm_ntohp (e->recv.buffer),
gm_ntohp (e->recv.message),
gm_ntohl (e->recv.length));
case GM_PEER_RECV_EVENT:
/* Handle data messages here in bounded time */
gm_provide_recv_buffer (my_port, some_buffer,
gm_ntohc (e->recv.size),
GM_LOW_PRIORITY);
break;
case GM_NO_RECV_EVENT:
/* Do bounded-time processing here, if desired. */
break;
default:
gm_unknown (my_port, e);
}
}
}
|
Any receive event not recognized by an application must be passed
immediately to gm_unknown(), as in the example above. The
function gm_unknown() will free any resources associated with the
event that the client application would normally be expected to free if
it recognized the type. Also, additional, undocumented event types will
be received by an application and are handled by gm_unknown().
These messages can be used for supporting features such as GM alarms and
blocking receives.
The motivation for putting small messages in the receive queue despite the fact that doing so might require a receive-side copy is the following set of observations:
gm_receive().
Therefore, placing small received messages in the receive command queue rather than in the more permanent receive buffer enhances performance and is worth the added complexity.
To prevent program deadlock, the client software must ensure that GM is
never without a receive token (buffer) for any potentially received
message for more than a bounded amount of time. Generally, except for
the case of message `forwarding' described in the next chapter,
this means that after each successful call to gm_receive() the
client will call gm_provide_receive_buffer() to replace the
receive token (buffer) with one of the same size and
priority before the next call to gm_receive() or
gm_send(). If such a deadlock condition exists for too long (on
the order of a minute) or too often (a significant fraction of a
one-minute interval), then remote sends directed at the receiving port
will time out.
| [ << ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |