Endianness and Socket Programming in C

I'm making a program that communicate with certain patient monitor using C sockets. I'm using connection-less sockets (UDP) to communicate with the device. But there is endianness mis-match between my computer and device and so far I was doing this to get parse response from the patient monitor:

recvfrom(int socket, char *buffer, size_t length, int flags,
             struct sockaddr *address, socklen_t *address_len);

Then I was casting the buffer directly to structure and using the ntohs and ntohl to change the byte ordering, for example:

struct A * a = (struct A *)buffer;
Struct A b;
b.v1 = ntohs(a->v1);
b.v2 = ntohl(a->v2);

After reading few examples over internet, I figured out that this may be wrong approach due to compiler dependent padding. But I'm not sure about it. I need simple way to un-marshall a buffer to a C structure with correct endiannes. The structure that I'm receiving can be of unpredictable length and little complex as well. I need fast and easy approach to do the un-marshalling.

I don't control sender. Sender is in network byte order. My question is only that:- Is is safe to cast a buffer to a structure and then use ntohs and ntohl on this casted structure to make host-byte order replica of this structure? Is it a fastest approach? If not, then what can be the fastest approach?

The answer depends on circumstances and specification of the patient monitor. Who writes the code for the patient monitor? Is there a specification for it? What machine architecture is it using(in case you know), are you dealing with just one model or are there many versions of the monitor -- can you change the monitor, etc. etc.

Im going to assume that you cannot change the monitor, and that the byte order is documented somewhere -- and you may have to create your own unpack/pack routines, by doing byte addressing and bit manipulation -- that is unless you know that the format exactly matches that of "network" order -- and that padding of structs are the same in the network buffer.

So something like;

void unpack(struct *b, unsigned char *buffer)
{
   b->v1 = (buffer[0] <<8)|(buffer[1]);   
   b->v2 = (buffer[2] <<24)|(buffer[3] <<16)|(buffer[4] <<8)|(buffer[5]);
   etc....
}   

or like this if you prefer to you ntohX;

void unpack(struct *b, unsigned char *buffer)
{
   b->v1 = ntohs(buffer+0);   
   b->v2 = ntohl(buffer+2);
   etc....
}   

However if you do control the monitor code, then using a tool like protocol buffers would get rid of all the complexity of doing bit manipulation and worry about byte orders....