New SdFat library (with and without DMA)

victor_pv Mon Apr 27, 2015 7:11 pm

***Latest Update 04/10/2017***
The code in the official repo from Bill Greyman has now the support for the libmaple based core (stm32duino) integrated in it. So no need to download it from my repo any more.

Official SD-Fat library repository:
https://github.com/greiman/SdFat

===============================================================

UPDATE TO THIS WARNING. SHOULD BE SOLVED IN THE LATEST REPO. If you have problems using pin 7 for CS, make sure you are in sync with Roger’s master STM32Duino repo.
WARNING!!
I was just testing an SD card with one of the sketches, and noticed that you CAN NOT use pin 7 in the maple mini (the stm32f103 native SS pin for spi1) for CS. I have not looked at how the library manages CS and how that pin is setup in out SPI library, but I guess there is a conflict in the pin mode. So the work around at the moment is to use a different pin for CS.
– Updated 06/04/2015-

– New update on 07/10/2015 – Small code changes to the library and specially to the SPI library, which should go to Roger’s master repo soon.

So as I posted before in the forum, I modified the latest SdFat library from Greiman.
I had submitted him changes for a previous version, but had issues. Now all the issues seem to be corrected in my latest version.
It is available here:
https://github.com/victorpv/SdFat

It seems to work fine for me for the few examples I have tested.
If you want the best performance for large transfers, can use DMA mode.

DMA would be advantageous for large transfers, but doesn’t help at all, and may even have a penalty, if you are just reading a few bytes to a log and things like that. Now if you transfer images or mp3 files from the sdcard, DMA can improve performance greatly, anything involving big blocks at once.

DMA can be enabled or disabled editing SdSpiSTM32F1.cpp, there is one #define line:
/** Use STM32 DMAC if nonzero */
#define USE_STM32F1_DMAC 1

Comment that line or change the 1 for 0, and DMA transfers will not be used.

RECOMMENDATIONS FOR SPEED:
Use DMA mode.
Set a bigger buffer, ideally 4KB

Read speeds without DMA, 4KB buffer:
read speed and latency
speed,max,min,avg KB/Sec,usec,usec,usec 676.29,8189,5927,6056 676.38,7467,5927,6054 676.38,7468,5927,6054 676.38,7469,5926,6054 676.38,7472,5926,6054

Replies20

RogerClark Wed Sep 14, 2016 8:49 pm

Re:SDIO

This may be possible with the Arduino Core that STM are developing.

I have asked them to put SDIO on their ToDo list, but at the momemt I dont know if they have the resources to create a SDIO lib for their core.

( see my announcement post in the announcements section)

stevestrong Thu Sep 15, 2016 7:34 am

Anyway, SDIO is not relevant for F103 family with less than 768kB flash, such as current blue and red pills.
For any other, specially Arch max, is more than welcome

Pito Thu Sep 15, 2016 8:05 am

Yea, SDIO is available on bigger chips only, also I’ve read somewhere ie. the 407 SDIO is buggy, the first well working on 446.
I did some naive experiments years back on 407 and chibios, I did 6MB/sec with 32kB blocks and a cheapo sdcard, not sure it was set up properly, however

Anyway, 3.5MB/sec with only a 512bytes large buffer on the MapleMini SPI is an impressive number, sure..
That is so close to the theoretical limit with 36MHz SPI clock.. Unbelievable :shock:

stevestrong Thu Sep 15, 2016 8:52 am

Pito Sat Sep 17, 2016 10:26 am

Hmm, write speed (3.5) higher than read speed (3.2) ? Is this really OK?
Anyway, just for the record, I found the SDIO bug discussion on ST forum.
Their benchmark: 2.8 write, 9.4 read using FatFS & SDIO 4 bit mode on STM32F4 disco.
The write speed is higher as the modern cards have got an large ram buffer inside, so the writes are buffered somehow.. (Info: BillG, author of the SdFat).
On Teensy 3.6 Bill did 19/20MB/sec with SDIO and the latest SdFat-beta, he claims:
https://forum.pjrc.com/threads/36737-Tr … post115239

PS: Mind he runs Teensy 3.6 @240MHz :shock:
https://www.kickstarter.com/projects/pa … -35-and-36

RogerClark Sat Sep 17, 2016 11:03 am

Pito wrote:

PS: Mind he runs Teensy 3.6 @240MHz :shock:
https://www.kickstarter.com/projects/pa … -35-and-36

Phono Thu Sep 22, 2016 7:31 pm

So let me jump into the bandwagon, and tell me the status of SDFat? Where can I find it? Is it now possible to select the SPI to be used, so that I can avail myself of the SD slot of the Olimex boards ?

Pito Fri Sep 23, 2016 10:28 am

https://github.com/greiman/SdFat-beta

You can assign what you want, I think it supports two SPIs with two cards in parallel now..

VadimEL Thu Sep 29, 2016 9:23 pm

Need example, replace chars in file.

I have file like
12345 434 3 fg23r 34f3 2g42h 34 3 h3443 f34 43f

stevestrong Fri Sep 30, 2016 7:15 am

– open the file as O_RD_WR
– file.rewind();
while(file pointer not at end) {
– peek the char (read pointer will not be incremented) – I think it is the function: file.peek();
– if (char==’3′) then: file.write(‘1’); // pointer will be incremented
– else: increment file pointer
}

VadimEL Fri Sep 30, 2016 1:38 pm

stevestrong thanks it’s work.

while (myFile.available())

{

check = myFile.peek();

if (check == '2')

{

myFile.write('1');

}

myFile.read();

}

myFile.close();

stevestrong Fri Sep 30, 2016 1:43 pm

Welcome

Manny Mon Oct 10, 2016 6:29 pm

Pito wrote:https://github.com/greiman/SdFat-beta
You can assign what you want, I think it supports two SPIs with two cards in parallel now..

RogerClark Mon Oct 10, 2016 8:58 pm

Manny

which file has the typo? and on what line number?

Manny Mon Oct 10, 2016 10:54 pm

RogerClark wrote:Manny

which file has the typo? and on what line number?

RogerClark Tue Oct 11, 2016 2:06 am

Ah OK

Not in my code then ;-)

stevestrong Tue Oct 11, 2016 9:14 am

Manny wrote:The typo is in SdFat-Beta, line 29 in SpiDriver/SdSpiSTM32F1.cpp

Manny Tue Oct 11, 2016 4:06 pm

stevestrong wrote:Manny wrote:The typo is in SdFat-Beta, line 29 in SpiDriver/SdSpiSTM32F1.cpp

SanjuPrakash Sun Oct 30, 2016 8:34 am

Hello.

I have tried executing most of the codes present in the victor_pv github page on my STM32F103C8T6 Minimum Development Board. I got SdInfo,Bench and PrintBenchmark working well. LowLatencyLogger compiles and uploads well but nothing gets displayed on the Serial monitor. Could you point out what could be going wrong?

And also could someone tell me what are the other SD codes that are running well on STM32F103C8T6 as i have been stuck on deciphering what to do to get the board to log data onto an SD card as fast as possible.

stevestrong Sun Oct 30, 2016 9:40 am

I suggest to use as basis the “RawWrite” example, this is suitable for fast SD card writing, with or without FAT support.
The latest SDFat-beta from greiman should also work, but not tried yet.
I am currently working with a modified version of it, DMA-based double-buffered analog acquisition and storage (while DMA writes the data from ADCs to one buffer, the other buffer being written onto SD card, I think I already posted somewhere an example).
I achieve ~350kBps (4 analog channels, dual simultaneous conversion using ADC1 and ADC2 in parallel, 12bits/channel, 44.1kHz sampling frequency, 1 second) without losing a byte, but without FAT support.
Now struggling to achieve fast read out speed from the card without DMA…