Spark Logic

Voice changer

racemaniac Fri Jun 10, 2016 6:09 pm

As i said in the 10$-o-scope project, i borrowed some of its code to do a fun little project

I read you can change the pitch of a voice by sampling sound into a circular buffer, and then reading from that buffer at a different rate, so no need for fourier transformations and difficult mathematics.
So i borrowed the DMA ADC code from the scope project (altered it a bit for this purpose), and connected a MP4725 DAC to my maple mini, and had some fun testing this little trick :)

How to wire it (on a maple mini):
I2C port 1 gets a mp4725 DAC
pin 3: analog input for the microphone
pin 4: analog input for a potentiometer that will alter the pitch

How it works:
I modified the dma code of this project to
– sample slower (about 35 khz)
– sample 2 pins simultaneously iso 1 pin interleaved
– run in circular mode
– only put the data of the sound sample pin into the memory (so i’m sampling 16 bits via dma, even though the 2 channels together are writing all 32 bits of the data register). I read the other pin directly from the dataregister when i need it.

The playback
– uses i2c clockstretching for a variable sampling rate
– tries to smooth out clicks (experiment with it, seems to work fairly well)

feel free to experiment with this fun little project :)
And let me know if you find better settings for buffer size & click removal

First thing that can be tried is a higher sampling rate, you should be able to push the DAC to 60khz with this code, and the sampling rate for the microphone can naturally also go a lot faster :).
Also better would be using a timer to send the samples :), but i just quickly wrote it with a delay :).

Also fun to try, program some special effects on it:
– maybe some echo?
– i read a dalek vocie is made by multiplying a sound signal by a sine wave (i think 100hz or something like that)
– feel free to add ideas

#include <dma_private.h>
#include <libmaple/adc.h>
#include <wirish.h>
#include <libmaple/i2c.h>

#define ADC_CR1_SIMULTANEOUS 0x60000 // simultaneous mode DUAL MODE bits 19-16
#define BufferSize 2048
uint16_t bufferPos = 0;
const int8_t analogInPin = PB0;
const int8_t analogInPin2 = PA7;

uint16_t dataBuffer[BufferSize];

const uint16_t clickTreshold = 100;
uint16_t lastSample = 2048;

uint16_t* analog2 = (uint16_t*)&(ADC1->regs->DR) + 1;

int modulationDelay = 40;
int wait = 0;

void i2c_str_InitPort(i2c_dev *device, uint8_t divider);
void i2c_str_StartSending(i2c_dev *device, uint8_t address);
void i2c_str_sendBytes(i2c_dev *device, uint8_t data, uint8_t data2);
void i2c_str_ReleasePort(i2c_dev *device);
void i2c_str_StopSending(i2c_dev *device);

#define I2C_STRETCH_SB_TIMEOUT 10000 //how many times we will check the status registers to see if the device is starting

#define I2C_STRETCH_DIV_400KHZ 30
#define I2C_STRETCH_DIV_600KHZ 20
#define I2C_STRETCH_DIV_800KHZ 15
#define I2C_STRETCH_DIV_1200KHZ 10

void setup()
{
Serial.begin();
pinMode(4, INPUT_ANALOG);
setADC();
startSampling ();
i2c_str_InitPort(I2C1, I2C_STRETCH_DIV_600KHZ);
i2c_str_StartSending(I2C1, 0x60);
}

void loop()
{
uint16_t sample = dataBuffer[bufferPos];
if((uint16_t)(sample - lastSample) > clickTreshold && (uint16_t)(lastSample - sample) > clickTreshold)
{
if(sample > lastSample)
sample = lastSample + clickTreshold;
else
sample = lastSample - clickTreshold;
}
i2c_str_sendBytes(I2C1, (uint8_t)(sample>>8), sample & 255);
bufferPos = (++bufferPos)%BufferSize;
lastSample = sample;
delayMicroseconds(20 + (*analog2>>7));
}

void startSampling ()
{
dma_init(DMA1);

adc_dma_enable(ADC1);
dma_setup_transfer(DMA1, DMA_CH1, &ADC1->regs->DR, DMA_SIZE_16BITS,
dataBuffer, DMA_SIZE_16BITS, (DMA_MINC_MODE | DMA_CIRC_MODE));// Receive buffer DMA
dma_set_num_transfers(DMA1, DMA_CH1, BufferSize);
dma_enable(DMA1, DMA_CH1);
}

void setADC ()
{
// const adc_dev *dev = PIN_MAP[analogInPin].adc_device;
int pinMapADCin = PIN_MAP[analogInPin].adc_channel;
int pinMapADCin2 = PIN_MAP[analogInPin2].adc_channel;
adc_set_prescaler(ADC_PRE_PCLK2_DIV_8);
adc_set_sample_rate(ADC1, ADC_SMPR_239_5);
adc_set_sample_rate(ADC2, ADC_SMPR_239_5);

// adc_reg_map *regs = dev->regs;
adc_set_reg_seqlen(ADC1, 1);
ADC1->regs->SQR3 = pinMapADCin;
ADC1->regs->CR2 |= ADC_CR2_CONT; // | ADC_CR2_DMA; // Set continuous mode and DMA
ADC1->regs->CR1 |= ADC_CR1_SIMULTANEOUS;
ADC1->regs->CR2 |= ADC_CR2_SWSTART;

ADC2->regs->CR2 |= ADC_CR2_CONT; // ADC 2 continuos
ADC2->regs->SQR3 = pinMapADCin2;
}

void adc_dma_enable(const adc_dev * dev) {
bb_peri_set_bit(&dev->regs->CR2, ADC_CR2_DMA_BIT, 1);
}

void adc_dma_disable(const adc_dev * dev) {
bb_peri_set_bit(&dev->regs->CR2, ADC_CR2_DMA_BIT, 0);
}

static void (*i2c_str_old_error_handler)(i2c_dev *dev);

void i2c_str_InitPort(i2c_dev *device, uint8_t divider)
{
i2c_peripheral_disable(device);
i2c_bus_reset(device);
i2c_init(device);//initialize it
i2c_config_gpios(device);//configure the gpios
device->regs->CR2 = I2C_CR2_ITERREN | 36; //dma enabled, peripheral frequency is 36Mhz
device->regs->CCR = I2C_CCR_FS | divider; //default 30
device->regs->TRISE = 11;
}
void i2c_str_ReleasePort(i2c_dev *device)
{
i2c_str_StopSending(device);
}

void i2c_str_StartSending(i2c_dev *device, uint8_t address)
{
i2c_peripheral_enable(device);//enable the port
i2c_start_condition(device);//set the start condition

uint32_t sr1 = device->regs->SR1;
uint32_t sr2 = device->regs->SR2;
uint16_t wait = 0;
while(!(sr1&I2C_SR1_SB))
{
if(wait++ > I2C_STRETCH_SB_TIMEOUT)
{
i2c_peripheral_disable(device);
return;
}
sr1 = device->regs->SR1;
sr2 = device->regs->SR2;
}
i2c_write(device, address<<1);//write the address of the device you want to contact (shifted 1 to the left)
}
void i2c_str_StopSending(i2c_dev *device)
{
i2c_stop_condition(device);
i2c_peripheral_disable(device);
}

bool i2c_str_IsSending(i2c_dev *device)
{
return !(device->regs->SR1 & I2C_SR1_TXE);
}

bool i2c_str_IsError(i2c_dev *device)
{
return device->regs->SR1 & (I2C_SR1_BERR | I2C_SR1_ARLO | I2C_SR1_AF | I2C_SR1_PECERR | I2C_SR1_TIMEOUT);
}

bool i2c_str_PortEnabled(i2c_dev *device)
{
return device->regs->CR1 & I2C_CR1_PE;
}

void i2c_str_sendByte(i2c_dev *device, uint8_t data)
{
uint32_t sr1 = device->regs->SR1;
uint32_t sr2 = device->regs->SR2;
if(sr1&(I2C_SR1_BTF | I2C_SR1_TXE | I2C_SR1_ADDR))
{
i2c_write(device, data);
}
}

void i2c_str_sendBytes(i2c_dev *device, uint8_t data, uint8_t data2)
{
uint32_t sr1 = device->regs->SR1;
uint32_t sr2 = device->regs->SR2;
if(sr1&(I2C_SR1_BTF | I2C_SR1_ADDR))
{
i2c_write(device, data);
i2c_write(device, data2);
}
}

Replies

ahull Sat Jun 11, 2016 2:08 am

I’ll need to have a play. I may adapt for one of the STM32F103 boards I have with built in DAC, or find my existing DAC board (whichever emerges from the clutter first). The STM32 as Vocoder sounds like a fun project. Exterminate.. exterminate… or maybe a touch of the Cylon voice from Battle Star Galactica. Bring on the heartless robots.

You might like to have a crack at digital ring modulation too, I think that is probably how the Daleks are synthesised.

RogerClark Sat Jun 11, 2016 7:51 am

I have several boards with F103R’s and F103V’s both of which have built in DAC’s. So it should be possible to output directly, without using any external hardware

I suspect the big limitation is RAM, as 20k on the F103C (and even the 64k on the F103V etc), is not going to give you a very long time delay buffer to implement things like echo.

The FFT project in another thread, may produce some interesting output audio, if the audio was reconstructed from the FFT. However I think the speed of the FFT is going to be to slow to make it usable for voice changing etc, though I suppose it may create some interesting effects

(Though I’m not precisely sure how you reconstruct a waveform based on a FFT)

racemaniac Sat Jun 11, 2016 9:52 am

RogerClark wrote:I have several boards with F103R’s and F103V’s both of which have built in DAC’s. So it should be possible to output directly, without using any external hardware

I suspect the big limitation is RAM, as 20k on the F103C (and even the 64k on the F103V etc), is not going to give you a very long time delay buffer to implement things like echo.

The FFT project in another thread, may produce some interesting output audio, if the audio was reconstructed from the FFT. However I think the speed of the FFT is going to be to slow to make it usable for voice changing etc, though I suppose it may create some interesting effects

(Though I’m not precisely sure how you reconstruct a waveform based on a FFT)

racemaniac Sat Jun 11, 2016 8:25 pm

I’ve now hooked it up to the microphone port of my pc, and started doing some more experiments with it :)
Now settled for a buffer size of 512 samples, which seems to work best :).
And i added a ring modulation to it, and it’s also working well :)

RogerClark Sat Jun 11, 2016 10:00 pm

Are you using a pre generated ring modulator input buffer, or generating it on the fly?

Re:FFT

i found this

https://www.dsprelated.com/showarticle/800.php

It looks like the forward FFT can be used to create an inverse FFT, wthout too much additional processing.

However, the speed of the FFT will be too slow to be usable for audio.

I was thinking that perhaps an interesting output from the FFT would be to sine wave frequencies and amplitudes of all the points on the spectrum analysis, and sum them together, (using the amplitudes from the spectum analysis)

But perhaps even this would take too long to process.

RogerClark Sat Jun 11, 2016 10:39 pm

On a related note…

I remember the “Specdrum” for the Sinclair Zx Spectrum computer.

https://www.dsprelated.com/showarticle/800.php

I remember, as a kid, working out that this just used a resistor ladder DAC.
So somehow I got hold of the software, and built my own simple DAC, at virtually no cost.

(I’ve no idea where I would have gotten the software from, as this was pre-internet and even pre modem; so perhaps someone I knew had the real hardware etc and let me “borrow” the program)

RogerClark Sun Jun 12, 2016 1:06 am

@racemaniac

I’ve taken a look at your code, but I don’t understand 2 things

The I2C code has a comments about DMA

device->regs->CR2 = I2C_CR2_ITERREN | 36; //dma enabled, peripheral frequency is 36Mhz

but as far as I can tell, you are not using DMA to output to I2C, you are calling i2c_write() from loop()?

You seem to be sub-sampling the input buffer into the output buffer using delayMicroseconds

Am I reading the code correctly?

racemaniac Sun Jun 12, 2016 3:27 am

RogerClark wrote:Are you using a pre generated ring modulator input buffer, or generating it on the fly?

Re:FFT

i found this

https://www.dsprelated.com/showarticle/800.php

It looks like the forward FFT can be used to create an inverse FFT, wthout too much additional processing.

However, the speed of the FFT will be too slow to be usable for audio.

I was thinking that perhaps an interesting output from the FFT would be to sine wave frequencies and amplitudes of all the points on the spectrum analysis, and sum them together, (using the amplitudes from the spectum analysis)

But perhaps even this would take too long to process.

RogerClark Sun Jun 12, 2016 4:22 am

Thanks

I should be able to easily modify the code to use the DAC on the F103V

racemaniac Sun Jun 12, 2016 7:20 am

RogerClark wrote:Thanks

I should be able to easily modify the code to use the DAC on the F103V

Pito Sun Jun 12, 2016 11:20 am

There is FFT and IFFT.
1. you must have an antialiasing filter in front of ADC, as well as some filtering at the DAC output
2. the messing with spectra for that purposes works usually:
a) you have got a ping/pong buffers
b) you do ADC into the ping one (via DMA), while in parallel you mess with FFT etc in the pong one
c) when the ping is full, DMA switches automatically the ADC stream to the pong one
d) then you do FFT etc in the ping one
e) and vice versa in a loop.
That is only way how to produce a continual onfly operation of ADC->FFT->something_do_with_spectra->IFFT->DAC
This is how it works with dspic33F.. microcontrollers. I doubt it is possible with F103.. But maybe there is a trick how to do it..
With 5000Hz sampling rate for “speech” you get 0.2secs to do FFT->something_do_with_spectra->IFFT-DAC.
Quite a lot of time ;)

RogerClark Sun Jun 12, 2016 11:28 am

@pito

I agree.

I doubt the F103 is fast enough to use FFT for realtime audio processing.

Kinda’ related…
I did a bit of research and found that STM have an application note about a vocoder using the F103, however I’m not sure they use the term vocoder to mean what I thought it meant.
The application note, discusses recording and compression of voice using the STM32

http://www.st.com/content/ccc/resource/ … 204907.pdf

And I eventually found the example code, but it will not work with the libmaple core (or I suspect the HAL MX core) as it uses the STM32 standard peripheral lib

Anyway, just for the record here is the download page

http://www.stmcu.org/document/download/index/id-203822

Pito Sun Jun 12, 2016 11:38 am

Speex – It seems to be compression only.

But back to Voice changer>
As I mentioned above, you get 0.2secs for playing with FFT-IFFT (1024point FFT, 5000Hz sampling rate, 2.5kHz speech bandwidth, ham radio quality).
That is a lot of time.
You do 1024FFT in 3msecs, the same time you need for IFFT.
So there is 0.194secs for playing with the Spectra (filtering, reversing, etc..).
If you master the ping/pong method with F103, you can do it..
If the RAM would be insufficient with 1024FFT, do it with 512 or 256, no big difference in quality.

RogerClark Sun Jun 12, 2016 11:47 am

@Pito

I thought that the 1024 pt fft was being computed in 2mS

So FFT + IFFT would be 4mS for 1024 (I will round this to 5ms, but I know this is optimistic)

If this was cut down to perhaps 32 point FFT, and had corresponding decrease in time taken, it would yield perhaps 32/1024 x 5ms = 164uS
Round to 200uS perhaps….

Or did I get my maths wrong ?

Pito Sun Jun 12, 2016 11:56 am

Again:
When you sample ADC into the ping buffer, you need 204.8ms to fill it full (5KHz sampling rate, 1024samples).
That is the TIME you have got available for the whole FFT->messingwithspectra->IFFT->DAC process within the pong buffer.
(Because you sample the ADC via DMA into the ping AND mess with FFT in pong in parallel).
A plethora of time..
Then you reverse (when the ping buffer is full – that is signalized by DMA flag or interrupt) – you start sample ADC into the pong buffer and do FFT-IFFT in the ping – again: you have got 204.8ms for playing with FFT->messingwithspectra->IFFT->DAC process within the ping buffer.
Without loosing a single sample..
Therefore it is called Ping-Pong.

The time available for FFT->messingwithspectra->IFFT->DAC process with the Ping-Pong method is:
(1/ADC_sampling_rate) * Number_of_FFT_points
Example:
1/5000*1024 = 204.8msecs

RogerClark Sun Jun 12, 2016 12:29 pm

OK

So, we already run the ADC on DMA, and could use DMA to output to either I2C DAC or internal DAC ( on the F103RC and larger MCUs in the family)

And use 2 buffers, i.e one filling and one being processed the being output.

Yes… 200ms is loads of time.

It would be interesting, just to run such a system e.g. 1024 point fft at 5khz sampling, and listen to what the reconstructed audio, of normal speach sounds like.

To accumplish a pitch change, I presume you could just move the data so that what was in bin[n] gets moved to bin[n+ pitch_shift] albeit this is granular.

Pito Sun Jun 12, 2016 12:38 pm

I can guarantee it will work :)
You can do with spectral lines what you want (with some limitations, subject to rocket science :) ).
You can make filters (by cutting the lines off – low, highpass, notch) or moving the lines up/down (to change the pitch], or reverse the spectra (to encrypt), or noise/denoise, mix, etc, etc. There are tons of books written about that matter :)

RogerClark Sun Jun 12, 2016 12:43 pm

@pito

Thanks

racemaniac Sun Jun 12, 2016 6:48 pm

Pito wrote:Again:
When you sample ADC into the ping buffer, you need 204.8ms to fill it full (5KHz sampling rate, 1024samples).

RogerClark Sun Jun 12, 2016 10:25 pm

I hadn’t realised that 35khz was the minimum sample rate.

But I think it should be OK, it just gives far less time to process the audio when in the Fourier domain. So rather than around 200ms, there would only be around 30mS

Input FFT takes 2mS for 1024 points. Output FFT will take about the same.

So that leaves around 25mS for processing 1024 points (512 pairs)

Which sounds like it should be enough to do some processing.

But, there is always a chance I overlooked something which kills the whole idea

Sign in to reply