Typecasting Pointers to different Datatypes - A Reflection

There were questions from multiple persons in the earlier article, all were about pointer typecasting and dereferencing. So I thought of writing about the same, let’s get started. Before that if you haven`t checked out those articles, please do so, the link is at the end of this article. It`s better to read this first before going through those links if you`re not familiar with pointer typecasting.

The most asked question is how this below code snippet ( and similar ) works?

uint32 u32Destination;

uint8 u8Source[ 12 ] ;

u8Source[ 0 ] = 0x01 ;

u8Source[ 1 ] = 0x23 ;

u8Source[ 2 ] = 0x45 ;

u8Source[ 3 ] = 0x67 ;

u32Destination  = *(( uint32_t * ) ( & u8Source [ 0 ] ) );

As you may know, I used this statement to copy the four byte value from long integer to the byte array contiguously. We can consider pointers which point to variables of different datatype to get to know how this works.

Before that let me start from basics, as you would have known pointer is just a ( integer ) variable which is used to hold the address of any other variable. To be precise it will have the address of the variable which it points to.

Let’s see what that means with the example. Consider the below code snippet,

uint32_t u32Integer = 0xAABBCC;

uint32_t * p32Pointer = & u32Integer.

Let’s assume the long integer variable u32Integer is located at 0x20000, then the pointer p32Pointer will be assigned the value of 0x20000 ( the address of u32Integer ). If you dereference the p32Pointer,  the controller will read four consecutive bytes from the location 0x20000 and will combine those four one byte values based on endianness of the architecture and will give you the four byte long integer value. So the statement ( * p32Pointer ) will give you the value of 0xAABBCC.

Let`s look at another example with a pointer to a short integer as shown below,

uint16_t u16Integer = 0xDDEE;

uint16_t * p16Pointer = & u16Integer.

For the sake of simplicity assume this u16Integer variable is also located in 0x20000, then the pointer p16Pointer will have the value of 0x20000 as well. In this case, if you dereference the p16Pointer then it will read two consecutive bytes from the location 0x20000 and will combine those two one byte values based on endianness of the architecture and will give you the two byte short integer value.

Note that there is no difference in the value ( address ) stored in the p16Pointer ( which is a pointer to a short integer ) and p32Pointer ( which is a pointer to a long integer ), both are having the value of 0x20000 which is the address of the variables u16Integer and u32Integer.

So during dereferencing, how the controller gives you the four byte value for p32Pointer and two byte value for p16Pointer? The difference is in the compiler interpretation while dereferencing the pointers. If the pointer points to the short integer then the compiler reads two bytes from the address the pointer points to. If the pointer points to the long integer then the compiler reads four bytes from the address the pointer points to.

So If you want to extract the lower two bytes from u32Integer and store it in uint16_t variable, you can achieve that with conventional AND method or typecasting, but you can also achieve the same using pointer as shown below,

uint16_t u16Integer = *( ( uint16_t * )  p32Pointer )

Have you got how this works? As we saw earlier, there is no difference in the value stored in the pointer variable, the pointer p32Pointer will have the value of 0x20000. By default, the compiler will interpret the p32Pointer as a pointer to the uint32_t as that`s what the datatype used in the declaration of p32Pointer. So if you want to read just two bytes from 0x20000, then you have to make that pointer as a pointer to uint16_t, this can be done through typecasting. So prefixing p32Pointer with ( uint16_t * ) will tell the compiler that the p32Pointer is a pointer to uint16_t, then the compiler will interpret the p32Pointer as a pointer to uint16_t.

Now if we dereference it, it will read two one byte values from the location 0x20000 and will combine those two bytes to give you the two byte short integer value. So as a result, u16Integer will have the value of  0xBBCC. Here you`ve to consider another subtlety, this result will come only in little endian based system, and will be different in big endian systems.

As I mentioned earlier, the value will be stored in memory based on the endianness of the system. For the example above, for the little endian based system, the value will be stored in memory as shown below,

0x20000 = 0xCC

0x20001 = 0xBB

0x20002 = 0xAA

0x20003 = 0x00

So in this case, if we read two bytes from 0x20000, the result we get is 0xCC and 0xBB, if we concatenate these two values based on little endian format the resultant two byte value is 0xBBCC.

So that`s what will be the value of *( ( uint16_t * )  p32Pointer ) statement.

Now let’s assume the endianness of the system architecture is big endian, the value will be stored in memory as shown below,

0x20000 = 0x00

0x20001 = 0xAA

0x20002 = 0xBB

0x20003 = 0xCC

So in this case, if we read two bytes from 0x20000, the result we get is 0x00 and 0xAA, if we concatenate these two values based on little endian format the resultant two byte value is 0x00AA.

So that`s what will be the value of *( ( uint16_t * )  p32Pointer ) statement.

I hope you understood how it works. Now let`s go back to the main question,

u32Destination  = *(( uint32_t * ) ( & u8Source [ 0 ] ) );

Here ( &u8Source[ 0 ] ) gives the address of the first element of u8Destination array. Let`s suppose it is located at 0x30000. Prefixing this ( &u8Source[ 0 ] ) with ( uint32_t * ) makes it the pointer to the unsigned long integer. So if you dereference it, the controller will read four one byte values starting from the location 0x30000 and will combine those values into one four byte value, which will be stored in u32Destination.

 So If you are working with little endian system and the buffer data is also aligned in little endian format, then you can use the dereference method discussed in my previous article. In big endian systems, your buffer data has to be aligned in the big endian format for this dereference method to work.

Refer the below link to know about the usage of dereferencing method,

Arul Prakash. “Pointer Hack in Packing & Unpacking the Frame” Embedded Playbook, Mar 2019.

Pointer Hack in Packing & Unpacking the Frame

Pointer is one of the tools in C which we can use to do whatever we want. There are many hacks we can do with the pointers. I thought of writing about one such hack that we can use and about the ways the same hack could backfire. Let’s get started.

Assume the following requirement, your project has more than one node and are interconnected that is communicating with one another. Let the communication line be anything ( LAN or serial ) , it`s obvious that the data is transmitted in the range of bytes. Each node is having data with various datatypes and want to share that data with one another. As the communication link is transmitting data as a byte, there comes the Packing and Unpacking in each node.

              

Suppose one node wants to share the array of data with datatype of uint32_t, then it should handle the conversion of data as shown below,

                             

uint32_t u32DataBuffer[10];

uint8_t u8LanTxBuffer[100];

                                            

u8LanTxBuffer[ 0 ] = u32DataBuffer [ 0 ] & 0xFF;

u8LanTxBuffer[ 1 ] = ( u32DataBuffer [ 0 ] >> 8 ) & 0xFF;

u8LanTxBuffer[ 2 ] = ( u32DataBuffer [ 0 ] >> 16 ) & 0xFF;

u8LanTxBuffer[ 3 ] = ( u32DataBuffer [ 0 ] >> 24 ) & 0xFF;

                                            

               And each node which receives the data, should handle the conversion as shown below,

              

uint32_t u32DataBuffer[10];

uint8_t u8LanRxBuffer[100];

                                            

u32DataBuffer[ 0 ]  = u8LanRxBuffer[ 0 ];

u32DataBuffer[ 0 ] |= u8LanRxBuffer[ 1 ] << 8;

u32DataBuffer[ 0 ] |= u8LanRxBuffer[ 2 ] << 16;

u32DataBuffer[ 0 ] |= u8LanRxBuffer[ 3 ] << 24;

                                            

                                            

Let`s suppose, you want to transfer 50 elements of u32DataBuffer, then there are two issues you will face. One is code readability as for the copy of one element you need four assignment statements and could take 200 line for the packing itself. The second issue is performance. 

Readability issue can be mitigated by employing for loop to iterate through or memcpy can also be used, but assume the worst case, the data you want to transmit in a single LAN packet consists of the assortment of different datatypes. Look at the below sequence,

              

LanDataTransmitBuffer <- uint32_t Data1

LanDataTransmitBuffer <- uint16_t Data2

LanDataTransmitBuffer <- uint8_t Data3

LanDataTransmitBuffer <- uint32_t Data4

   

                          

In above case for loop can`t be used to iterate through and you need to use four statements for single copy of uint32_t data and it will really mess up the code readability. This issue can be fixed by using macro as shown below,

              

#define UINT32_TO_UINT8_IN_LE( destination, source )      \

do                                                                                             \

{                                                                                                \

destination[ 0 ] = source & 0xFF;                                   \

destination[ 1 ] = ( source >> 8 ) & 0xFF;                       \

destination[ 2 ] = ( source >> 16 ) & 0xFF;                     \

destination[ 3 ] = ( source >> 24 ) & 0xFF;                     \

}while( 0 )

                                            

The macro can used as shown below,

                                            

UINT32_TO_UINT8_IN_LE( LanDataTransmitBuffer[ 0 ] , Data1 );

Code readability issue is fixed, but what about the performance, this packing and unpacking in each node surely consumes considerable amount of time in the total transmission as for each copy there are four load/store instruction in addition to the shifting and other instruction. What can be done for this?

This is where one of the pointer hack can be used to improve the performance. The assignment can be done using a single statement by using pointers instead of the above method where at least four statements are needed. Below single statement can be used to do the same copy of uint32_t data into the byte buffer array as above method,

                             

*(( uint32_t * ) ( &destination[ 0 ] ) ) = source[ 0 ];

                                            

                                            

Fair and simple right? Yes you can`t type these many things for each conversion and having this for each conversion would make code a bit unreadable and makes it prone for mistake, this can be fixed with a simple macro definition as shown below,

                             

#define UINT32_TO_UINT8_IN_LE( destination, source )                \

( *(( uint32_t * ) ( & ( destination ) ) ) = ( source ) )

                                        

    

The macro can used as shown below,

                                            

UINT32_TO_UINT8_IN_LE( LanDataTransmitBuffer[ 0 ] , Data1 );

                                

                           

Hurrah, We`ve achieved what we want in a single statement instead of four statements ( Similar macro can be implemented for unpacking in receiving node ). Performance wise is this is considerable amount of improvement, Code readability wise also it`s okay.

                             

Is there anything wrong with this method? Can this be used on any system blindly without any other consideration?

                                            

As with the usual cases of using pointers, there`s one loop hole here also and that could create havoc if you don`t take necessary precaution.

                                 

           

Any guess what it is? Yes the issue is Unaligned memory access, In Higher end processors which supports unaligned memory access this pointer dereference method can be used without any fuss, But as most of the embedded system consists of low or medium end processor which may or may not support unaligned memory access this is definitely a worrying issue.

                                            

One way of tackling this problem is taking care of the alignment of uint8_t data buffer while creating it, this can be done by using pragma. By allocating the starting byte of uint8_t data buffer in the address which is multiple of four as needed by our controllers, we can sort out the issue. But the problem with this method is, if there`s requirement which needs mixture of data with different datatype as mentioned previously, this will fail, as we may do uint32_t data copy from buffer element located in address which is not a multiple of four. So In the processors which doesn`t support unaligned memory access this method can`t be used.

                     

        

In most processors which supports unaligned memory access there are certain things you need to ensure before using this method. First one is to ensure the Processor MMU is properly configured to support unaligned memory access.

For example, in ARM7 Architectures, you need to disable the unaligned memory access trap in CP15 register before using the above method. Likewise you need to look for the appropriate configuration in other architectures as well.

                             

You also need to ensure another thing, you should define the uint8_t data buffer in the memory region which is not a strongly ordered memory region in Cache Lookup table. If you`ve configured that memory region as strongly ordered, then again you`ll end in trap.

                             

So there are important things to consider even in the processor`s which support unaligned memory access. With these many configurations using this method will definitely painful in the software which may need be ported to different platforms and architectures in future. Another  basic thing you need to consider with this method is endianness.

                             

With these many things to consider do you think this can be used in production software, especially in safety critical systems? Well I`ve seen this method used in production code of Class III Medical Device ( yes, software failure will result in death of the person). That device uses Intel Atom processor and VxWorks platform. As these higher end processors platforms supports unaligned memory access, we`ve used this pointer dereference method in the project. If you know what you`re doing, then with Pointers you can make your software run with its utmost efficiency, but even if you miss tad a bit you`ll have to face the wrath.

              

Okay, what`s your take? Will you go for this pointer deference method or traditional method or memcpy? If you`ve any other alternative, please let me know in comments or by mail.

Porting lwIP for embOS

This is my second article. In this, I`m going to write about the experience of porting lwIP for embOS platfrom and SAMA5D3x architecture. As you know porting anything to different platform is always challenging as any simple mistake could cost you quite a lot of debugging time, porting TCP/IP stack is much more challenging.

 Please note that this is not how to do article, in fact I`m not even going to give the definition of all the necessary functions need to be written for lwIP stack, as it could make this blog a mini book.

 To port lwIP for embOS or any other platform, we need to provide the definitions for the functions as required by the lwIP stack ( function list can be found in the reference link ). Below are the definitions for some of those functions in the list. 

sys_sem_new:

               In this function, we need to create semaphore for the stack and need to return the ID of the newly created semaphore ( or address depends on your definition of sys_sem_t ). As embOS needs the address of the OS_RSEMA variable each time ( which should be unique), we need to find whether the given OS_RSEMA variable is used or not, simple way to do that is having one structure containing OS_RSEMA variable and the status flag indicating whether the corresponding semaphore is used or not. We will loop through the array and find the unused semaphore and return its address. We also need to change the flag status as used. If the count is zero, we need to take the semaphore, that is done by OS_Use function call, so that task requiring semaphore would be blocked till the semaphore is released by the stack. ( In all the coming function definitions, returning error code can be handled better with single return statement. For time being I`ve written multiple return statements within single function )

typedef OS_RSEMA* sys_sem_t;

typedef struct _semWrapper

{

   OS_RSEMA tcpSemaphore;

   uint8_t usedFlag;  

}semWrapper;

static semWrapper tcpSemArray[MAX_SEM_STRUCT];

sys_sem_t sys_sem_new(u8_t count)

{

   uint8_t locLoopCount = 0;

  

   for(locLoopCount = 0;locLoopCount < MAX_SEM_STRUCT;locLoopCount++)

   {

      if(!(tcpSemArray[locLoopCount].usedFlag))

      {

         break;

      }

   }

   if(locLoopCount >= MAX_SEM_STRUCT)

   {

      return NULL;                                               /* TBD - Debug Print to be added - msarul */

   }

   sys_sem_t  xSemaphore = &(tcpSemArray[locLoopCount].tcpSemaphore);

   tcpSemArray[locLoopCount].usedFlag = 1;

  OS_CreateRSema(xSemaphore);

   if(count == 0)                                                /* Means it can't be taken */

   {

      OS_Use(xSemaphore);

   }

   return xSemaphore;

}

sys_sem_free:

               Here we will loop through the array to find the match of the given semaphore and we will delete the same using OS_DeleteRSema function and we will change the flag status into unused.

void sys_sem_free(sys_sem_t sem)

{

   uint8_t locLoopCount = 0;

  

   for(locLoopCount = 0;locLoopCount < MAX_SEM_STRUCT;locLoopCount++)

   {

      if(sem == (&(tcpSemArray[locLoopCount].tcpSemaphore)))

      {

         break;

      }

   }

   if(MAX_SEM_STRUCT <= locLoopCount)

   {

      return;                /* TBD - Debug Print to be added. Shouldn`t come here, msarul */

   }

   tcpSemArray[locLoopCount].usedFlag = 0;

   OS_DeleteRSema( sem );

}

sys_arch_sem_wait:

               We need to wait for the semaphore to be released, in case if timeout is provided and if it is elapsed we need to return the mentioned error ( SYS_ARCH_TIMEOUT ). Otherwise we need to return the elapsed time.

u32_t sys_arch_sem_wait(sys_sem_t sem, u32_t timeout)

{

   int startTime, endTime, elapsed;

  

   startTime = OS_GetTime();

  

   if( timeout != 0 )

   {

      if( OS_UseTimed( sem, timeout ) != NULL )

      {

         endTime = OS_GetTime();
         if ( endTime >= startTime )
         {
            elapsed = ( endTime - startTime );
         }
         else
         {

            elapsed = ( endTime + ( 0xFFFFFFFF - startTime ) + 1 ) ;
         }

         if( elapsed == 0 )

         {

            elapsed = 1;

         }

         return (elapsed);

      }

      else

      {

         return SYS_ARCH_TIMEOUT;

      }

   }

   else     /* must block without a timeout */

   {

      OS_Use(sem);

      endTime = OS_GetTime();

      if ( endTime >= startTime )
      {
         elapsed = ( endTime - startTime );
      }
      else
      {

         elapsed = ( endTime + ( 0xFFFFFFFF - startTime ) + 1 ) ;
      }

      if( elapsed == 0 )

      {

         elapsed = 1;

      }

      return ( elapsed );

   }

}

sys_sem_signal:

               OS_Unuse function releases the provided semaphore, so that task blocked on the same can continue its execution.

void sys_sem_signal(sys_sem_t sem)

{

   OS_Unuse( sem );

}

sys_mbox_trypost:

               In this function we will put the address stored in the given message pointer in the message queue. One important thing to note here is we are not passing the given data pointed by specified pointer through message queue, we`re passing the content of the pointer ( which is address of the message ) through message queue, so during fetching we need to take care of this, will see that in explanation of fetch function. If you look at embOS manual about OS_Q_Put function, it says argument pSrc should point to the message to store. So the statement “ OS_Q_Put(mbox,&msg,4)) ” causes the address stored in the given pointer message ( msg ), passed as data in the message queue.

err_t sys_mbox_trypost(sys_mbox_t mbox, void *msg)

{

   if(NULL == OS_Q_Put(mbox,&msg,4))

   {

      return ERR_OK;

   }

   else

   {

      return ERR_MEM;

   }

}

sys_arch_mbox_tryfetch:

u32_t sys_arch_mbox_tryfetch( sys_mbox_t mbox, void **msg )

{

   void *dummyptr = NULL;

   unsigned int * desPointer = NULL;

  

   if( msg == NULL )

   {

      msg = &dummyptr;

   }

  

   if(NULL != OS_Q_GetPtrCond( mbox, (void *)(&desPointer)) )

   {

      *msg = (void *)(*desPointer);

      OS_Q_Purge(mbox);

      return ERR_OK;

   }

   else

   {

      return SYS_MBOX_EMPTY;

   }

}

               Before going into the explanation, let look at embOS manual to know about OS_Q_GetPtrCond function. It says argument ppData of OS_Q_GetPtrCond should point to the address of the pointer which will be set to the address of the message. As we have seen earlier, we have passed the content of pointer as data in sys_mbox_trypost function, so the OS_Q_GetPtrCond function will set the desPointer with the address of the memory location where the passed data ( which is the pointer content – the address of the original data ) is located. So if you dereference desPointer, it will give the address of the original message, that is after dereference, it has the address of the original message. So we can store this value with *msg. I think with example we will be able to understand this better.

Example:

               In api_msg.c file of lwIP source code ( You can download the source from reference link ), inside the accept_function, there exists one pointer to netconn structure as shown below,

struct netconn *newconn;

              Memory is allocated using netconn_alloc function for this. This pointer is put into mailbox using the below statement.

if (sys_mbox_trypost(conn->acceptmbox, newconn) != ERR_OK) {

As explained above, function sys_mbox_trypost will put the address stored in the pointer newconn in the message queue, rather than data present in the address pointed by the pointer newconn.

The message queue data is received in netconn_accept function located in api_lib.c file. Here the netconn pointer is declared as shown below,

struct netconn *newconn;

            The data from the mailbox is received by the below statement inside the netconn_accept function,

if (sys_arch_mbox_fetch(conn->acceptmbox, (void *)&newconn, conn->recv_timeout) == SYS_ARCH_TIMEOUT) {

As you can see here, address of pointer newconn is passed as **msg argument in sys_arch_mbox_fetch function, so the argument **msg should be pointing to the valid data ( which is netconn structure data passed in accept_function function ) after dereferencing it two times. That means after dereferencing it one time ( *msg ) it should point to the address of the netconn structure data. If you see the definition of sys_arch_mbox_tryfetch, after dereference desPointer pointer has the address of the original message ( netconn structure data ), so it just can be assigned to *msg as shown below.

*msg = (void *)(*desPointer);

This much complexity wouldn`t be there if we just put the structure data itself in message queue data instead of pointer to that structure data. But to share the memory created by netconn_alloc, we`re doing this.

  

Other than the above mentioned functions all other function definitions required by lwIP like sys_arch_mbox_fetch, sys_mbox_post , sys_thread_new, etc can be implemented as all of them are similar to this.

Issues:

Okay we`ve implemented all the function definitions required by lwIP. Now let`s look at the issues that could come in our way. There`re many issues that could come. Let`s look at some of them.

One of the main issue that could come is with Cache configuration. You possibly know about cache coherency issues involved here as DMA handles the data transfer of LAN packets, which operates independently of CPU. The simple solution to that is to put DMA Descriptors and buffers in non-cacheable memory region. The procedure for that depends on the compiler used. One of the common mistake we could make here is forgetting to configure the corresponding memory region in linker configuration file. For example in the existing project some name could`ve been used to assign the memory region as non-cacheable and the same would`ve been configured in liker file, so it would work. But in the new project we`re porting, the linker configuration file could have been assigned a different name for non-cacheable memory region or in worst case no memory region at all is assigned as non-cacheable, all of the memory is configured as cacheable. So we`ve to configure the memory region as non-cacheable in both c file as well as in linker configuration file.

               There`s one more configuration related to cache that could cause the issue. As you`re using different project with different RTOS. There`re chances it would`ve different MMU Table set up procedure. There`s high possibility that RTOS initialization function will set up MMU table and in that the memory region you have assigned as non-cacheable could`ve been configured as cache-able. So it`s not enough to check the configuration in DMA descriptor file and in linker configuration file you need to check the MMU table set up in your initialization as well. Because that`s what the final say goes to the controller about the memory region from your source.

Other than this if there are hardware changes, there are possibilities of changes in Phy device address as it depends on the voltage levels on the PHYAD pins. This is the case if there`s small or no change in PHY Device. If the PHY device variant gets drastically changed than you need to take care of the configurations related to that as well, even though IEEE defined registers remain same, there`re good number of PHY registers which are vendor-specific, you need to take care of that as well.

After resolving above issues and many more issues you could face, finally you will be able to communicate with the target. I`ve faced almost all of the issues. Unfortunately for me PHY device was also different one and it had one vendor specific user register and that made me breaking my head for almost an entire day. Hope your porting succeeds. In case you`ve any comments please let me know.

Have a look at below links for some reference:

                         1) lwIP Porting for an OS                          

                         2) lwIP Source

                         3) embOS User Manual