Struggling to Extract Accurate CIR Data from DWM1000 Modules - Need Help!

jogi · November 3, 2023, 9:26am

Hello everyone,

I hope you are all doing well. I am writing to seek your assistance after days of persistent efforts to extract accurate Channel Impulse Response (CIR) data from Accumulator Memory using DWM1000 modules. Unfortunately, I have yet to achieve the desired results. I have been working with the ss_TWR examples for CIR data extraction in a Line-of-Sight (LOS) environment. To test the modules, I’ve placed them in an unfurnished room at various distances (1m, 2m, 3m, and 5m).

My primary headache right now is that I’m unable to read more than 254 samples from the Accumulator Memory. Any attempt to read 255 samples results in the memory returning zeros. Moreover, when I attempt to read 1016 samples (64MHz Pulse Repetition Frequency), it causes a complete system freeze. Even when I read just 254 values, the amplitude fluctuates dramatically, sometimes falling below 500 or spiking above 6000.

Furthermore, even when I manage to read the first 254 samples, I notice that the “firstPath” index (from diagnostics) appears at around 740 to 750 index, indicating that I might only be reading noise from the memory. It’s worth noting that I’ve come across research papers mentioning similar issues with the “firstPath” index around 740. Photo is taken from one of paper.

I have a few questions that I hope you can help me address:

How can I successfully read all 1016 samples with accuracy?
How can I leverage the dwt_readdiagnostics fields to improve my CIR data extraction code?

Here is my current implementation, which includes reading the receiver frame quality diagnostic values and processing the data:

// Read the receiver frame quality diagnostic values
dwt_readdiagnostics(&rx_diag);

// Print the diagnostic values
printf("maxNoise: %u\n", rx_diag.maxNoise);
printf("firstPathAmp1: %u\n", rx_diag.firstPathAmp1);
printf("firstPathAmp2: %u\n", rx_diag.firstPathAmp2);
printf("firstPathAmp3: %u\n", rx_diag.firstPathAmp3);
printf("stdNoise: %u\n", rx_diag.stdNoise);
printf("maxGrowthCIR: %u\n", rx_diag.maxGrowthCIR);
printf("rxPreamCount: %u\n", rx_diag.rxPreamCount);

// Calculate the fractional and integer part of firstPath
// ... (code for firstPath calculation)

uint16_t ACCUM_DATA_LEN = 253;
uint8 accumulator_data[ACCUM_DATA_LEN];
uint16 bufferOffset = 0;
dwt_readaccdata(accumulator_data, ACCUM_DATA_LEN, bufferOffset);

int i = 1;
for (i = 1; i < ACCUM_DATA_LEN; i += 4) {
    // ... (code for processing and printing complex values)
    nrf_delay_ms(1);
}
free(accumulator_data);

I would greatly appreciate any insights, suggestions, or guidance you can offer.
Best regards,
Jalal

PS: I have went through FAQ but could not found answer!

AndyA · November 6, 2023, 9:19am

Did you look back in the forum? There are lots of posts on reading the CIR data if you search for them.
Personally I don’t use the stock drivers but over people have had no issues reading the full data using them. Or you can always read the data is several blocks using the offset value.
Remember that there are 6 bytes per CIR value so if you want all 1016 values you need to read 6096 bytes. Plus the first byte received is junk so 6097 bytes. If reading in several blocks then don’t forget to allow for the first byte of each block being junk. If you only read 253 bytes then that’s only 42 values. If you are assuming a read of 253 gives you 252 values then you’re probably getting some buffer overrun bug which is what is causing the crash.

You then also need to to correctly handle converting the 3 byte, 18 bit values to 32 bits. For negative numbers this requires you to set the top 14 bits. Failing to do this will make a small (as in close to zero) negative number appear to be a large positive one.

Generally the first path will be between around index 740 and 750. This is simply an artifact of how the chip works internally. There are some nasty multipath situations which can result in a first path being found far earlier.

jogi · November 6, 2023, 4:03pm

Hello AndyA,

Thank you for your prompt response. I am currently working with a pair of dwm1001 modules, and I believe each CIR value consists of 5 bytes. The first byte is a dummy, and the remaining four bytes represent the CIR value, with the first two bytes being the real part and the other two bytes being the imaginary part. Please correct me if I’m mistaken.

I’ve also attempted to read the accumulator memory in blocks, but unfortunately, I haven’t been successful. I’m using the example codes ss_twr_init for the initiator and ss_twr_resp for the responder within the Segger Embedded Studio IDE. I’ve added the following code snippet to “ss_init_main.c.”

    uint16 ACCUM_DATA_LEN = 254; // Complex value samples
    uint16 totalSamples = 1016; // Total number of samples you want to read
    // Process all four blocks in a loop
    for (int block = 0; block < totalSamples+1; block += 254) {
      uint8* accumulator_data = (uint8*)malloc(ACCUM_DATA_LEN); // Allocate memory for each block
      if (accumulator_data == NULL) {
            // Handle memory allocation error
            printf("Memory allocation failed\n");
            return 1;
      }
        // Calculate the correct offset for the current block
        uint16 offset = block * ACCUM_DATA_LEN;

        // Read accumulator data for the current block
        dwt_readaccdata(accumulator_data, ACCUM_DATA_LEN, block);

        size_t accumulator_data_size = sizeof(accumulator_data);
        printf("Accumulator size in no. of bytes are: %d \r\n", accumulator_data_size); // getting always 4
        size_t num_elements = sizeof(accumulator_data) / sizeof(uint8);
        printf("Accumulator size in no. of elements are: %d \r\n", num_elements); // getting always 4

        // Process data in the current block
        for (int i = 1; i < ACCUM_DATA_LEN; i += 4) {
            uint8 lowRealPart = accumulator_data[i]; // Index correction to skip dummy byte
            uint8 highRealPart = accumulator_data[i + 1];
            uint8 lowImaginaryPart = accumulator_data[i + 2];
            uint8 highImaginaryPart = accumulator_data[i + 3];

            // Combine low and high 8-bit values for real and imaginary parts (Little Endian)
            int16 decimalRealPart = (int16)((highRealPart << 8) | lowRealPart);
            int16 decimalImaginaryPart = (int16)((highImaginaryPart << 8) | lowImaginaryPart);

            // Print the real and imaginary parts
            printf("%d, %d \r\n", decimalRealPart, decimalImaginaryPart);

            nrf_delay_ms(1);
        }
    free(accumulator_data);
    }

I’m still struggling to grasp the concept behind the offset value. The manual mentions that setting the offset value to zero should read the entire memory, but in my case, it doesn’t seem to work as expected. Additionally, I’ve experimented with reading by specifying a specific sample value (as shown in the code above), but I haven’t had much success despite spending several days on it.

Need lead! Thank you
regards,
jogi

AndyA · November 8, 2023, 9:34am

Sorry, you are correct, the values are 16 bits in the DW1000, it’s the DW3000 that has the 18 bit values.
So yes, 4 bytes per value plus 1 dummy one at the start of the read.

You do however have some bugs in your code:

e.g. sizeof(accumulator_data) will always give 4 on a 32 bit machine. You are asking for the size of the pointer.

Personally I’d structure the code a bit more like this:

    int BlockSize = 254; // samples per block
    int totalSamples = 1016; // Total number of samples you want to read

    // 4 bytes per sample
   //allocate outside the loop, memory allocation is slow
	uint8* accumulator_data = (uint8*)malloc(BlockSize * 4 + 1); // Allocate memory for each block
    if (accumulator_data == NULL) {
        // Handle memory allocation error
        printf("Memory allocation failed\n");
        return 1;
    }

    // offset in samples not bytes
	for (int offset = 0; offset < totalSamples; offset += BlockSize) {

                // calculate how many samples to read
		int samples = BlockSize;
		if (samples + offset > totalSamples) // less than one block left
		{
           samples = totalSamples-offset;
		}

        // Read accumulator data for the current block - read 4 bytes per sample required. Starting at the first required samples address
        dwt_readaccdata(accumulator_data, samples*4+1, offset*4);

//        size_t accumulator_data_size = sizeof(accumulator_data);
//        printf("Accumulator size in no. of bytes are: %d \r\n", accumulator_data_size); // getting always 4
//        size_t num_elements = sizeof(accumulator_data) / sizeof(uint8);
//        printf("Accumulator size in no. of elements are: %d \r\n", num_elements); // getting always 4

        // Process data in the current block
        for (int i = 0; i < samples; i++) {
            uint8 lowRealPart = accumulator_data[i*4+1]; // Index correction to skip dummy byte
            uint8 highRealPart = accumulator_data[i*4+2];
            uint8 lowImaginaryPart = accumulator_data[i*4+3];
            uint8 highImaginaryPart = accumulator_data[i*4+4];

            // Combine low and high 8-bit values for real and imaginary parts (Little Endian)
            int16 decimalRealPart = (int16)((highRealPart << 8) | lowRealPart);
            int16 decimalImaginaryPart = (int16)((highImaginaryPart << 8) | lowImaginaryPart);

			// or assuming a little endien processor you can simplify to:
			int16 decimalRealPart = *((int16*)(accumulator_data + i*4+1));
			int16 decimalImaginaryPart = *((int16*)(accumulator_data + i*4+3));

            // Print the sample number, real and imaginary parts
            printf("%d, %d, %d \r\n", offset+i, decimalRealPart, decimalImaginaryPart);

            nrf_delay_ms(1);
        }
    }
    free(accumulator_data);

Each register address is effectively a separate block of memory. Offset is the address within that block of memory to start reading/writing.

jogi · November 8, 2023, 4:00pm

Hello AndyA,

Thank you very much!
It helped me a lot and now I am able to extract CIR samples.

Best regards,
jogi