Noise question.

[Teleno]

My PI design has a preamplifier gain of 17 an an ADC resolution (after oversampling/decimation at 64 samples) of 16 bits with a 1.1V reference (Atmega328P).

I'm getting low frequency noise spikes (0.5 - 1 Hz) of 250 ADC value. This translates to about 4mV output noise and 235uV input noise.

Is this average or below the standards of the popular PI projects?

The ADC runs on a 500 kHz clock, so the effective resolution is 8 bits.

[Carl-NC]

If I understand, you have an 8b ADC you are oversampling by 64x. If so, that produces an ideal 11b, not 16b. You get 1/2 bit every time you double the oversampling. So 1.1V/2^11 = 0.54mV LSB.

Regardless, 4mV sounds a bit high. You would need to post a schematic of the front-end in order to see if this is justified.

[Teleno]

If I understand, you have an 8b ADC you are oversampling by 64x. If so, that produces an ideal 11b, not 16b. You get 1/2 bit every time you double the oversampling. So 1.1V/2^11 = 0.54mV LSB.


Regardless, 4mV sounds a bit high. You would need to post a schematic of the front-end in order to see if this is justified.

The ADC is 10 bit although at 500KHz only the 8 MSBs are reliable, but I'm using all 10 nevertheless.


Oversampling at 64 gives me 3 extra bits, but then I pass this to an IIR filter with a tau of 64 samples which gives me another 3 extra bits. Tx frequency is 1Khz.


This is what the relevant code looks like:

#define RING_SIZE 64
#define RING_PRECISION 3   // 3 extra bits from 64 samples
#define IIR_PRECISION 3    // 3 extra bits from IIR with 64 tau


typedef struct {
   uint16_t buff[RING_SIZE];
   uint32_t sum;
   uint16_t decimated;
   uint16_t average;
   uint8_t  index;
} RingBuffer;


void update_ring_buffer (RingBuffer * rb, uint16_t value){
    rb->sum += value;
    rb->sum -= rb->buff[rb->index];
    rb->buff[rb->index] = value;
    rb->decimated = rb->sum >> RING_PRECISION; // extra 3 bits here
    rb->index++;
    if (rb->index >= RING_SIZE)  rb->index = 0;
}

boolean EFReady = false;
volatile uint16_t targetValue;
uint32_t targetFilteredValue = 0, thisTarget = 0;


RingBuffer targetRing = {{0},{0}, 0, 0, 0};

void main() {

   /* Initialize IIR filter. targetRing.decimated is 13 bit */

   targetFilteredValue = targetRing.decimated << 6;  // Fixed point arithmetic 26.6
.
.
.
  while (1) {      
      while (!EFReady);      // wait for an acquisition cycle to complete (Earth Field sample ready)    
      EFReady = false;
                  
      update_ring_buffer(&targetRing, targetValue); // targetValue is 10 bit, provided by he ADC interrupt      

      /* IIR low pass filter, gain 1, tau ~ 64 sampling periods. */
           
      targetFilteredValue= targetFilteredValue - (targetFilteredValue >> 6) + targetRing.decimated;

      thisTarget =  targetFilteredValue >>  IIR_PRECISION;   // final value with 16 bits 
.
.
. 
}

If I set the threshold at 80 the noise triggers the audio every 2 or 3 seconds (short beep), which is livable and the detector becomes usable.
In input terms it would be a threshold of 80 * (1.1V / 65535) / 17 = 79uV.

So I'm adding 6 bits and code 80 is 6.3 bits... hm... perhaps this noise is the effect of not excluding the 2 LSB thrashed by the 500kHz ADC clock ???

Regarding the schematic, it is here: https://www.geotech1.com/forums/show...213#post295213

[Teleno]

I've read about the noise in ADCs having 3 components: 1/f noise, white quantization noise and phase noise peaking at the sampling frequency. The noise I'm getting looks like 1/f, so I guess I won't be able to get rid of it.

In any case a noise threshold of 80 uV doesn't look too bad, I can detect a 0.7g gold nugget at 8 - 9 cm at 6.5 us delay, even thoygh I'll probably never find one.

[F117]

My math knowledge is a little bit rusted, so I am just trying to understand what you implemented here - triggered by my simple ring buffer implementation in the Pickini project.
I assume this initialization is not needed, since it is outside the main loop and targetFilteredValue is already initialized to 0:

* Initialize IIR filter. targetRing.decimated is 13 bit */

targetFilteredValue = targetRing.decimated << 6; // Fixed point arithmetic 26.6

Not sure why you are not using the ring buffer average value ( sum / 64 ) instead of rb.decimated = (sum / 8 )...

Suppose the ring buffer contains steady values = 64 times 128.
The next 2 values are 256 instead of 128:

1. ring buffer [ 64 x 128 ]
value = 256
rb.sum = 8320 --> average = 130
rb.decimal = 1040
// Assume targetFilterValue is still 0 ( it wouldn't be as there are already 64 samples in the ring buffer...)
targetFilteredValue = 0 - 0 + 1040
thisTarget = 1040 / 8 = 130 // The same value as the buffer average

2. ring buffer [ 63 x 128 and 1 x 256 ]
value = 256
rb.sum = 8448 --> average = 131
rb.decimal = 1056
targetFilteredValue = 1040 - 16 + 1056 = 2080
thisTarget = 2080 / 8 = 260 // Almost double the buffer average

3. ring buffer [62 x 128 and 2x256 ]
value = 256
rb average = 134
thisTarget = 390



In the Pickini firmware and - if I recall correctly - also in the examples in the book "Inside the metal detector", this very slow running average is used as reference ( ground value).
So far my 2cts - maybe a more elaborate simulation with values ( in excel ? ) would reveal why there are excessive values ?

[Teleno]

My math knowledge is a little bit rusted, so I am just trying to understand what you implemented here - triggered by my simple ring buffer implementation in the Pickini project.
I assume this initialization is not needed, since it is outside the main loop and targetFilteredValue is already initialized to 0:


Not sure why you are not using the ring buffer average value ( sum / 64 ) instead of rb.decimated = (sum / 8 )...

Suppose the ring buffer contains steady values = 64 times 128.
The next 2 values are 256 instead of 128:

1. ring buffer [ 64 x 128 ]
value = 256
rb.sum = 8320 --> average = 130
rb.decimal = 1040
// Assume targetFilterValue is still 0 ( it wouldn't be as there are already 64 samples in the ring buffer...)
targetFilteredValue = 0 - 0 + 1040
thisTarget = 1040 / 8 = 130 // The same value as the buffer average

2. ring buffer [ 63 x 128 and 1 x 256 ]
value = 256
rb.sum = 8448 --> average = 131
rb.decimal = 1056
targetFilteredValue = 1040 - 16 + 1056 = 2080
thisTarget = 2080 / 8 = 260 // Almost double the buffer average

3. ring buffer [62 x 128 and 2x256 ]
value = 256
rb average = 134
thisTarget = 390



In the Pickini firmware and - if I recall correctly - also in the examples in the book "Inside the metal detector", this very slow running average is used as reference ( ground value).
So far my 2cts - maybe a more elaborate simulation with values ( in excel ? ) would reveal why there are excessive values ?

I didn't post the code to make it short but the ring buffer is indeed initialized, the initial targetFilteredValue is not 0 but the sum of 64 samples less three LSBs.

I didn't use the average because by decimation theory the sum of 64 samples has 3 bits more resolution than a single sample, but the average throws away the extra bits.

The IIR filter tau is a power of 2 so multiplications and divisions can be faster done by shifting left and right. In this case the tau is 64 samples which produces 90% of the full target value after 2.3 tau periods. At 1kHz it's 2.3 x 64 x 1 ms = 147 ms. At a sweeping speed of 0.5 m/s this 90% of the full signal occurs after 7.4 cm of target exposure.

I assumed a slow sweeping speed because this PI is intended for detecting nuggets on bedrock, so no large distances are to be covered in a short time. Also, there's a 1KHz higher limit to the TX frequency otherwise the Atmega328P at 16MHz cannot complete acquisition, filtering, sound and display within a Tx period.

I've also tried a 16 sample ring buffer and a 16 tau IIR filter, each providing an increase of 2.5 bits for a total of 15 bits (10 ADC + 2.5 + 2.5). This allows for faster sweeping with a 90% target response at 2.3 x 16 x 1 ms = 36.8 ms. The sensitivity is halved but it's compensated because for the same sweep speed you can get 99% response after only 8cm of target exposure, less than half the diameter of the coil. Also a threshold of 20 gets rid of the shot or 1/f noise for a sensitivity of 40uV, so in the end the PI performs better.

[Qiaozhi]

For anyone wanting to understand how oversampling and decimation works, here's a very good explanation (attached).

Teleno - I haven't looked at your code yet as I'm on holiday this week.

[Teleno]

The 15-bit version is much more stable.

Here's the 15-bit signal after EF and background subtraction.

The two largest peaks in each graph are the signals of a 0.7g nugget at 7cm (top graph) and at 6 cm (bottom). The rest is the noise.

One tick of the Y axis represents 2uV at the preamp's input. (Vref / resolution) / Gain = (1.1V / 32768 ) / 17.

For anyone wanting to understand how oversampling and decimation works, here's a very good explanation (attached).

Teleno - I haven't looked at your code yet as I'm on holiday this week.

I dont'want t spoil your vacation . Enjoy!

[Teleno]

For anyone wanting to understand how oversampling and decimation works, here's a very good explanation (attached).

Oversampling is based on noise in the samples having a gaussian distribution. I think when oversampling at 1000 Hz the noise below 500 Hz cannot get cancelled because it doesn't change much between samples. The flicker noise gets amplified instead.

[waltr]

I am thinking your NOISE issue may be due to environment not hardware or code.
Most of us have issues with noise in our workshops with PI detectors (and other detector designs) due to the amount of EMI dues to AC Mains (including data on these lines) plus all the other electronics devices.

Try taking it outside away from all Mains and electronics to see if these noise spikes go away.

[Teleno]

I am thinking your NOISE issue may be due to environment not hardware or code.
Most of us have issues with noise in our workshops with PI detectors (and other detector designs) due to the amount of EMI dues to AC Mains (including data on these lines) plus all the other electronics devices.

Try taking it outside away from all Mains and electronics to see if these noise spikes go away.

Actually I go often to the beach and I see no difference in the noise, it's still there. It's random rather than periodic
. The coil is shielded.

[KingJL]

@Teleno, not to hijack this thread or interfere with your design, have you considered the use of audio ADC's. The are (relatively when compared to other 24 bit ADC's) inexpensive and have some impressive performance specs. Mdtoday and I have been doing some design work using the 24 bit CS53xx Audio ADC's (initial design using XS5340). The oversampling frequency is ~6.5 MHz which depending on your final desired sample rate, results in oversampling factor that ranges from 32:1 to 3174:1 (initial design using 2048 sample rate resulting in 3174:1 oversampling factor).

[Teleno]

@Teleno, not to hijack this thread or interfere with your design, have you considered the use of audio ADC's. The are (relatively when compared to other 24 bit ADC's) inexpensive and have some impressive performance specs. Mdtoday and I have been doing some design work using the 24 bit CS53xx Audio ADC's (initial design using XS5340). The oversampling frequency is ~6.5 MHz which depending on your final desired sample rate, results in oversampling factor that ranges from 32:1 to 3174:1 (initial design using 2048 sample rate resulting in 3174:1 oversampling factor).

I appreciate your tip. I know some of you are using these ADCs and I'm sure to great advantage. I stick to an MCU because I want my PI to be minimalistic (two op-amps, two 555s an Atmega328P, two 3-pin regulators and the associated passives) and based on generic parts yet getting the most out of each part.

It's a beautiful sunny day here in The Netherlands so I've just been to the beach in Scheveningen with a 15 bit version of the PI firmware (as above). It performed very well on a threshold of 20 - 25, better than the 16 bit version at 80 threshold. There were some random short beeps every 5 - 10 seconds but nothing bothering.

Alas, findings were mostly beer caps down to 20 cm deep (the diameter of the spiral flat coil is 15 cm, 170 uH, 1.8 A) and nails. The only coin I found (50 euro cents) was lying there in plain sight asking to be picked up. I found it because I went detecting but not with the detector )).

By the way, the spiral coil combined with pitch modulation is very good at pinpointing too.

[Carl-NC]

I'm not sure I understand. You get 3 bits from oversampling+filtering, but you don't get 3 bits from oversampling and another 3 bits from filtering. I suspect you have ~12 ENOBs in the end.
I assume you've disconnected the coil and the noise is still there?
Unfortunately the specs for the ADC are very minimal. But if you're getting 4mV of noise spikes at the input you should be able to see that on an oscope. If you see them at the output but not the input then look at the ADC reference.
Offhand I don't see anything in the schematic that raises a flag, but I'm not sure I understand it either.

[Teleno]

I'm not sure I understand. You get 3 bits from oversampling+filtering, but you don't get 3 bits from oversampling and another 3 bits from filtering. I suspect you have ~12 ENOBs in the end.
I assume you've disconnected the coil and the noise is still there?
Unfortunately the specs for the ADC are very minimal. But if you're getting 4mV of noise spikes at the input you should be able to see that on an oscope. If you see them at the output but not the input then look at the ADC reference.
Offhand I don't see anything in the schematic that raises a flag, but I'm not sure I understand it either.

My assumption is that filtering is equivalent to oversampling because it consists of adding samples. Although the samples were previously oversampled, they contain noise, so the oversampling theory also applies to the filtered samples.

When you add "targetFilteredValue - (targetFilteredValue >> 6) + targetValue" the result has 6 bits more (the initial targetFilteredValue is the decimated ADC sample shifted 6 times left). After 5x tau the filter has added enough data to overcome the initial value and it's equivalent to adding 64 times the new value to the initial one (98% new value) so it has 3 more bits accuracy. From then on you get 16 bit samples with 5 tau delay plus the delay of the ring buffer.

Actually if I just do oversampling and skip filtering (or viceversa) the sensitivity drops dramatically, so I believe it's working.

The diptrace schematic isn't nice but on the same thread you have the LTSpice schematic of the analog part which is much more friendly.