Very Computer

Sending (and receiving) Ethernet frames via raw packet socket...

Is there any way to force automatic (limited number of times) retransmission
of frame (packet) when transmit is failed?

Otherwise, is there any way to get to know if the frame is actually
transmitted or not?

Regards,
    Vadim

Quote:> Sending (and receiving) Ethernet frames via raw packet socket...

> Is there any way to force automatic (limited number of times)
retransmission
> of frame (packet) when transmit is failed?

> Otherwise, is there any way to get to know if the frame is actually
> transmitted or not?

> Regards,
>     Vadim

Well, its a very interresting question . If could check the stats for
the given interface if you dont have much traffic on the network .
If the sendto(2) return a value < 0, then the packet has not been sent
but sometimes sendto return a positive value and the packet is wrong .

if you find an accurate way to check it, i'm interested .

Sent via Deja.com http://www.deja.com/
Before you buy.

Quote:>Well, its a very interresting question . If could check the stats for
>the given interface if you dont have much traffic on the network .
>If the sendto(2) return a value < 0, then the packet has not been sent
>but sometimes sendto return a positive value and the packet is wrong .

As far as I understand from source code write() always completes
successfully if packet is enqueued to network device. But the problem is
that packet sockets are always asynchronous and write() completes when
packet is enqueued and does not wait for corresponding interrupt from
hardware therefore loosing any completion status.

Checking for statistics does not seem very elegant to me.

Quote:>if you find an accurate way to check it, i'm interested .

In my understanding there is no accurate solution if people stay with
existing implementation. BTW: I checked 2.2.4, simply because my RH use it.
Could it be any improvement in latest version? I do not think so. It would
require significant amount of work.

I asked just to make sure that I have proper vision of network support in
Linux.

Yes. Just use a (PF_INET,SOCK_STREAM) socket.

Quote:> Otherwise, is there any way to get to know if the frame is actually
> transmitted or not?

Not possible without a protocol that sends acks, so no.

Watching interface error counters is not sufficient, because it does not
guarantee that the packet reached the application layer at the other end
(remember IP/Ethernet packets are allowed to be dropped when convenient, e.g.
when the host is temporarily out of memory)

-Andi

Quote:>Yes. Just use a (PF_INET,SOCK_STREAM) socket.

I deal with ethernet frames completely composed by application.

Quote:>Not possible without a protocol that sends acks, so no.

The problem is that the ACK is lost. Unfortunatley I do not understand how
to detect that that it was not sent (carrier check failed). Because Linux
does not seem to tell you that. And I have to know to be able to organize a
retransmit or to tell the application that it is not sent.

It must be possible because ANY ethernet adapter tells for sure whether the
frame is transmitted or not, how many collisions detected and so on. I do
not want to know if the peer managed to receive - I want to know if the
packet is sent and if not I would like to know why (carrier check, too many
collisions).

Just to answer some questions: I'm building emulator of old fashion Ethernet
adapter.

Quote:>Watching interface error counters is not sufficient, because it does not
>guarantee that the packet reached the application layer at the other end
>(remember IP/Ethernet packets are allowed to be dropped when convenient,
e.g.
>when the host is temporarily out of memory)

I do not talk about IP. On hardware level I can check if the frame is sent
or not. So I want to be able to detect the same from user level application.

Regards,
    Vadim

Quote:

>-Andi

ACKs get lost. That's life. Network protocols must to operate
properly when ACKs are lost.  It doesn't matter how/why they're
lost.  If your system breaks down when an ACK is lost, then
you're protocol is broken.  No amount of tweaking the OS will
fix it.

If you're not operating with a 100% reliable physical layer
(and you're not), then ACKs can get lost and there may be no
way to detect it.  Your protocol has to tolerate packet loss.

Quote:>Because Linux does not seem to tell you that.

There are many ways in which an ACK can get lost.  Most are not
detectible.  Notifying the user of one but not all of them is
useless.

Quote:>And I have to know to be able to organize a retransmit or to
>tell the application that it is not sent.

If you have to know whether your ACK frame was lost or not,
then your protocol is broken.

Quote:>It must be possible because ANY ethernet adapter tells for sure
>whether the frame is transmitted or not,

Nobody cares whether a frame is transmitted or not.  People
care wheter data gets to the application at the far end.
That's what transport protocols are for.

Quote:>how many collisions detected and so on. I do not want to know
>if the peer managed to receive - I want to know if the packet
>is sent and if not I would like to know why (carrier check, too
>many collisions).

That's a very odd statement.  You don't care if a frame is
received by anybody, just that it's transmitted.  If that is
truely the case, then just always assume that it's transmitted.

Since there is no difference between a frame being transmitted
and not received and a frame not being transmitted, then you
may assume whichever case you like.

Quote:>I do not talk about IP. On hardware level I can check if the
>frame is sent or not. So I want to be able to detect the same
>from user level application.

I expect that would involve significant work on Ethernet driver
and network stack.  Most importantly it would not buy you
anything.  ACKs could still get lost even thought he Ethernet
board thinks they've been transmitted.  Your protocol has to
deal with that.

--
Grant Edwards                   grante             Yow!  Look DEEP into the
                                  at               OPENINGS!! Do you see any
                               visi.com            ELVES or EDSELS... or a
                                                   HIGHBALL??...

I understand all your arguments and believe me I'm not a newbee in network
programming.

If you have time, please look into any of Ethernet driver. There you will be
able to see that certain device registers give you an indication of TX
status. I want to know if that status is OK or not. Nothing more.

Quote:>There are many ways in which an ACK can get lost.  Most are not
>detectible.  Notifying the user of one but not all of them is
>useless.

Can not agree on this. Some information which you call "useless" may help a
lot detecting source of problem. And if you hide information it makes
troubleshooting process much more difficult (have you ever got a call from
customer saying "nothing works"?). This is why devices usually have status
registers (see above). This is why I have to emulate status register of that
Ethernet controller for which I provide software emulation.

Quote:>Nobody cares whether a frame is transmitted or not.  People

Device driver does. At least in OpenVMS, and in Linux, and in any other
operating system.

Quote:>Since there is no difference between a frame being transmitted
>and not received and a frame not being transmitted, then you

The difference is the status provided by Ethernet controller on TX_DONE
interrupt.

Quote:>anything.  ACKs could still get lost even thought he Ethernet
>board thinks they've been transmitted.  Your protocol has to
>deal with that.

Totally agree. It has to, but surely it does not. Anyway I do not care about
that protocol. I did not design it. But I just want to distinguish two
situations: packet is not sent because cable is removed from TP socket or
broken; or packet is sent but get lost somewhere in hubs. In addition it
would be nice to get to know how many collisions had been detected before
packet is sent.

Regards,
    Vadim

Quote:

>--
>Grant Edwards                   grante             Yow!  Look DEEP into the
>                                  at               OPENINGS!! Do you see
any
>                               visi.com            ELVES or EDSELS... or a
>                                                   HIGHBALL??...

No you can't. That was the whole point. The linux ethernet layer honours
the IP internetworking principle that states "packets can be dropped anywhere
when convenient" (mostly in low memory situations). Also standard 802.2
ethernet does not even have a way to do guaranteed delivery (there are
IEEE 802.x protocols for reliably delivery, but they're not directly
accessible via Linux packet sockets except when you implement them yourself)

For an reliable protocol you need at least L3 acknowledgements.

-Andi

Agreed -- all of the Ethernet controllers I've seen made that
information available.  The problem is that it's extremely
difficult (with current stack/driver designs) to get that
status info back up to the application.  At the point where the
Tx failure status is known, the driver typically doesn't know
to whom the failed packet belongs.

[...]

Quote:>>Since there is no difference between a frame being transmitted
>>and not received and a frame not being transmitted, then you

>The difference is the status provided by Ethernet controller on
>TX_DONE interrupt.

Right, but in either case, an application has to do the same
thing.  If the action taken by the app is the same for two
cases, then it can be argued that it's not required that the
appliction know which case occured, only that one of them
occured.

Quote:>>anything.  ACKs could still get lost even thought he Ethernet
>>board thinks they've been transmitted.  Your protocol has to
>>deal with that.
>Totally agree. It has to, but surely it does not.

That's too bad.

Quote:>Anyway I do not care about that protocol. I did not design it.
>But I just want to distinguish two situations: packet is not
>sent because cable is removed from TP socket or broken; or
>packet is sent but get lost somewhere in hubs.

That information could, in theory, be passed back up to the
application, but it's a pretty * problem.  The send() or
write() syscall returns as soon as the information has been
queued for transmit.  At some point later in time, the Ethernet
driver notices that a transmit failed.  Now there are two
problems:

 1) The ethernet driver doesn't know who requested the packet
    be sent.

 2) If it did, there isn't a well-defined way (other than a
    signal) to inform an application of an asynchronous error
    like that.

Quote:>In addition it would be nice to get to know how many collisions
>had been detected before packet is sent.

If you just want to query the state of the interface, that's a
considerably easier problem.  Doing a cat on /proc/dev/net
shows fields for both collisions and carrier-faults (among
other things).

I don't know if there is a more direct API to get that info,
but open() and read() on /proc/net/dev/ should get you that
info.

--
Grant Edwards                   grante             Yow!  Bo Derek ruined
                                  at               my life!
                               visi.com            

Yes. Actually this is exactly what I see looking into kernel. I know this is
just a design issue of any UNIX. Sometimes I see that UNIX is quite well
designed but there are some solutions  which could not be easily implemented
in any UNIX platform. BTW: In the situation we are talking about, Windows
NT/2000 is much more convenient: it associates a request with every packet
sent via similar packet interface, so that you can synchronize execution
with completion of requests processing and obtain status of transmission: OK
or NOT.

And of course I do not want anybody to rewrite the whole Linux kernel. I
just asked if there is something in Linux which I do not see. Looks like
there is no "undocumented" feature which would help me.

Quote:>If you just want to query the state of the interface, that's a
>considerably easier problem.  Doing a cat on /proc/dev/net
>shows fields for both collisions and carrier-faults (among
>other things).

Yes, and this is what I do now. But the problem is that I have to know when
my packet is processed by hardware. For the moment I just wait for any
change in TX statistics and then I check which counter is changed. But I can
not agree this is a reliable solution. Given the fact that Linux is quite
"liquid" who guarantees that some clever driver writer does not forget to
update TX statistics? If there is no accepted standard interface, there is
no guarantee.

Quote:>I don't know if there is a more direct API to get that info,

Loks like there is no one.

Quote:>but open() and read() on /proc/net/dev/ should get you that
>info.

>--
>Grant Edwards                   grante             Yow!  Bo Derek ruined
>                                  at               my life!
>                               visi.com

This is what I suspected but just asked to be sure.

Quote:>For an reliable protocol you need at least L3 acknowledgements.

Quite agree.

Regards,
    Vadim

Vadim,

I recently encountered an equivalent problem, which it probably
wouldn't hurt to be aware of: when injecting packets using the
PF_PACKET socket interface, it is possible for the driver packet queue
to become full, at which point packets queued at the packet scheduling
layer are quietly dropped.

Since the environment I was working could guarantee that only one
process was using the ethernet device at a time, I crufted up a /proc
driver that simply dumped the packet drop statistics from the packet
scheduling layer:

  for(pdev = dev_base; pdev && result < MAXFILELEN; pdev = pdev->next){
    result += sprintf(page + result, "%s", pdev->name);
    if(pdev->qdisc){
      result += sprintf(page + result,
"\t%i\n",pdev->qdisc->stats.drops);
    }
    else{
      result += sprintf(page + result, "*\n");
    }
  }

Whilst this doesn't tell you _which_ packets have been dropped, it at
least lets you know that _some_ have been dropped.  It's far from
perfect, but perhaps you could extend your ethernet driver to count
TX failures and do the same sort of thing i.e. tell you that _some_
packets have failed to transmit.

I guess the full solution to your problem would be to implement a char
driver interface to your ethernet card and have non-blocking writes
which would return error on TX failure, bypassing the entire socket
layer.  Of course, this may be overkill for your current requirements.

Guy.

Actually it does, otherwise it wouldn't be able to manage the socket
send buffer. It'll just not help the original poster because there is no error state passed
to the destructor callback.
(but then it is useless anyways, because successfull TX says nothing alltogether
about successfull arrival even at the link layer)

Quote:>  2) If it did, there isn't a well-defined way (other than a
>     signal) to inform an application of an asynchronous error
>     like that.

Linux 2.2 has one. See ip(7), MSG_ERRQUEUE. It is only feed by ICMP messages currently.

-Andi

Quote:>Actually it does, otherwise it wouldn't be able to manage the socket
>send buffer. It'll just not help the original poster because there is no

error

So If you allow synchronous write() on such socket, it becomes in principle
possible to return -1 and set errno to something?

Actually I did something very similar in principle. After write() to raw
packet socket I poll /proc/net/dev file waiting for TX statistics update. If
some TX counter is changed then you can decide what had happened depending
on which counter is changed. It is not elegant, but looks like the only
solution provided that I can not (and actually do not want to, and have no
time to) change anything within the kernel.

If you develop a product for Linux platform it is not acceptable to supply
any patches to kernel to make the product working. IMHO.

Anyway, thank you very much for information. At least I known I'm not alone.

Vadim