Spark Logic

New SdFat library (with and without DMA)

victor_pv Mon Apr 27, 2015 7:11 pm

***Latest Update 04/10/2017***
The code in the official repo from Bill Greyman has now the support for the libmaple based core (stm32duino) integrated in it. So no need to download it from my repo any more.

Official SD-Fat library repository:
https://github.com/greiman/SdFat

===============================================================

UPDATE TO THIS WARNING. SHOULD BE SOLVED IN THE LATEST REPO. If you have problems using pin 7 for CS, make sure you are in sync with Roger’s master STM32Duino repo.
WARNING!!
I was just testing an SD card with one of the sketches, and noticed that you CAN NOT use pin 7 in the maple mini (the stm32f103 native SS pin for spi1) for CS. I have not looked at how the library manages CS and how that pin is setup in out SPI library, but I guess there is a conflict in the pin mode. So the work around at the moment is to use a different pin for CS.
– Updated 06/04/2015-

– New update on 07/10/2015 – Small code changes to the library and specially to the SPI library, which should go to Roger’s master repo soon.

So as I posted before in the forum, I modified the latest SdFat library from Greiman.
I had submitted him changes for a previous version, but had issues. Now all the issues seem to be corrected in my latest version.
It is available here:
https://github.com/victorpv/SdFat

It seems to work fine for me for the few examples I have tested.
If you want the best performance for large transfers, can use DMA mode.

DMA would be advantageous for large transfers, but doesn’t help at all, and may even have a penalty, if you are just reading a few bytes to a log and things like that. Now if you transfer images or mp3 files from the sdcard, DMA can improve performance greatly, anything involving big blocks at once.

DMA can be enabled or disabled editing SdSpiSTM32F1.cpp, there is one #define line:
/** Use STM32 DMAC if nonzero */
#define USE_STM32F1_DMAC 1

Comment that line or change the 1 for 0, and DMA transfers will not be used.

RECOMMENDATIONS FOR SPEED:
Use DMA mode.
Set a bigger buffer, ideally 4KB

Read speeds without DMA, 4KB buffer:
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
676.29,8189,5927,6056
676.38,7467,5927,6054
676.38,7468,5927,6054
676.38,7469,5926,6054
676.38,7472,5926,6054

Replies

robca Sun Dec 20, 2015 10:35 pm

I need help in figuring out what I’m doing wrong.

I have a standard SD Card reader, which I tested with a 3.3V Arduino Pro Mini: it works with a Patriot test card, both using the Arduino SD library and the newer SDFat one

I can’t make it work on the Maple Mini. According to viewtopic.php?f=28&t=37&start=10#p723 I connected MOSI to pin4, MISO to 5, SCK to 6. I tried both 7 and 10 as CS, with no difference.

I’m using a new download of the Master repository (hence I have the latest SPI files), and downloaded and installed as library Victor’s port of SdFat (https://github.com/victorpv/SdFat). I’m using the code below:

#define SD_CS_PIN 10
#include <SPI.h>
#include <SdFat.h>
SdFat SD;

File myFile;

void setup() {
// Open serial communications and wait for port to open:
Serial.begin(9600);

pinMode(7, OUTPUT);
pinMode(10, OUTPUT);

if (!SD.begin(10)) {
Serial.println("initialization failed!");
return;
}
Serial.println("initialization done.");

// open the file. note that only one file can be open at a time,
// so you have to close this one before opening another.
myFile = SD.open("test.txt", FILE_WRITE);

// if the file opened okay, write to it:
if (myFile) {
Serial.print("Writing to test.txt...");
myFile.println("testing 1, 2, 3.");
// close the file:
myFile.close();
Serial.println("done.");
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}

// re-open the file for reading:
myFile = SD.open("test.txt");
if (myFile) {
Serial.println("test.txt:");

// read from the file until there's nothing else in it:
while (myFile.available()) {
Serial.write(myFile.read());
}
// close the file:
myFile.close();
} else {
// if the file didn't open, print an error:
Serial.println("error opening test.txt");
}
}

void loop() {
// nothing happens after setup
}

Manny Sun Dec 20, 2015 11:09 pm

I would use a schematic for the board attached to work out which pins go to what header.

stevestrong Mon Dec 21, 2015 7:56 am

I would use SCK divider in SD init, because the default one may be to high:
#define CS PA4
sd.begin(CS, SPI_CLOCK_DIV8)

robca Mon Dec 21, 2015 4:16 pm

Thanks Steve.

I tried pretty much all the divisor values possible, but still won’t initialize (this is similar to a problem reported in another thread). The PA4 vs Pin 7 doesn’t seem to make any difference, either

I connected a Saleae clone with SPI protocol, and I can see that the Arduino board uses a slow signal. But even if I divide the clock enough to be in the same range as the Arduino, the card still won’t initialize. I can see that chip select correctly goes low both using PA4 and pin 7

On the other hand, I can use the standard Arduino SD card library. What I noticed is that the initialization routine for the old card uses a very slow clock (~250kHz), while the new library (even when using 128 as a divider in SdFatConfig.h) uses ~500kHz. I can actually use the full SPI speed with my Class6 card when reading and writing with the old library (at a full 2.5MHz)

Lastly I noticed that there is a SdSpiSTM32F1.cpp file in the library, but the library always uses SdSpiCard.cpp and doesn’t seem to use the F1 version (I added Serial.print statements in all functions, and none was used). Is that a configuration error on my side?

Do you mind sharing the SdFatConfig.h file you are using?

My experience is similar to previous reports, where some people can use the standard SD card library, but not the new one… I wonder if there’s still a bug somewhere, in the SPI implementation (I have checked, btw, and my SPI code is identical to the one in Victor’s repository)

As an aside: what do people use to debug? I have an STLink, and would prefer debugging thru it, than having to add serial.print everywhere

stevestrong Mon Dec 21, 2015 5:04 pm

@robca,
I am using the default files of the SdFat lib, didn’t make any change.
I spent a lot of time in order to make SPI2 work, but honestly I didn’t spend to much time to work with SPI1, only shortly checked and it worked.
However, u should consider to check out the “development” branch, as there are some sensitive changes committed regarding SPI needed by SdFat to work on SPI2, and it may be relevant also for SPI1.

The file SdSpiSTM32F1.cpp should be in use, for example the function init(). The rest of functions are mostly thin wrapper for calling functions of SPI.cpp.
Finally, if nothing else helps, please check the selected board type in Arduino.
Is your board a Baite-clone or an original one?

RogerClark Mon Dec 21, 2015 9:17 pm

Try the development branch of the repo, instead of the Master

The development branch has Steve’s latest fixes, but I have not had time to merge the development back into Master and test it..

robca Mon Dec 21, 2015 9:34 pm

stevestrong wrote:@robca,
I am using the default files of the SdFat lib, didn’t make any change.
I spent a lot of time in order to make SPI2 work, but honestly I didn’t spend to much time to work with SPI1, only shortly checked and it worked.
However, u should consider to check out the “development” branch, as there are some sensitive changes committed regarding SPI needed by SdFat to work on SPI2, and it may be relevant also for SPI1.

The file SdSpiSTM32F1.cpp should be in use, for example the function init(). The rest of functions are mostly thin wrapper for calling functions of SPI.cpp.
Finally, if nothing else helps, please check the selected board type in Arduino.
Is your board a Baite-clone or an original one?

zmemw16 Tue Dec 22, 2015 2:35 am

daft idea, but could this manifest itself as display flicker on the tft ili9341 displays i’ve been trying to get to work?
i get a white screen, with the shade of white changing in sync with test mode changing.
the working one, now one and a bit as something waggled with a second and it showed bits of the test display with much flickering.
i was beginning to think it might be timing in the init sequence for the controller.

stephen

stevestrong Tue Dec 22, 2015 12:43 pm

doesn’t belong this topic to another thread? It can maybe related to SPI, but not to SdFat lib.

zmemw16 Tue Dec 22, 2015 12:55 pm

apols, i agree.
i’ll re-post in problems with libraries.

it was the mention of ‘tight’ timings and the cs signal being applied earlier that woke me up.
anyways i off to get the devel branch and try it out … …

stephen

racemaniac Sun Apr 17, 2016 6:21 pm

I’ve recently been playing around with DMA to offload my cpu for an IO intensive project i’m working on, and so far wrote some small libraries of my own so when the DMA is running, i can do other work while it’s doing its thing.

I’ve also been playing around with this library, and was wondering if we could make it even more powerful than it already is by allowing you to do other things while the DMA is working.

My simplest thought so far is: what if you can pass a method it has to loop for you while you’re waiting for DMA (could be the main loop method of your arduino project). It hardly takes any changes at all to the library (just allow the user to set that method, and put the call to that method in the loops where you’re waiting for the DMA to finish, and you’re done). This may not be the most efficient way of doing it, but you’ll probably get 80% of the result for 1% of the work :).

Ideally you would start a call, and it would do its thing in the background, and use interrupts to see state changes & react on them, and you can periodically poll it to see if it’s done yet and then read your result, but that would be a big redesign of the library :). And it probably won’t gain that much compared to the simple solution :).

rexnanet Mon Jun 13, 2016 10:05 am

e-dredon wrote:Good afternoon all.

I didn’t have any success yet with the SD module of my ILI9341 LCD display, and the optimized SdFat library.

Now with my STM32F103C8T6 (bluepill)
I tried EVERY digital pins for the SD_CS line.
I’ve disabled the DMA optimization for initial tests in the config file SdSpiSTM32F1.cpp
#define USE_STM32F1_DMAC 0

I’ve tried to use SPI1 for both the ILI9341 display and SD module, with a dedicated CS pin for the SD_CS.
I’ve tried the latest SPI library with the new SPI.setModule(2); in the setup section of my code. In that case I hooked the SD miso/mosi/sck on the SPI2, and specified the SD_CS (I tried the SS of the SPI2 and other pins like 0, 1 because I’ve seen some users got success with a mapple mini with that configuration)
Last but not least, I’ve tried various DIV_CLOCK settings (2, 4, 8) with no success too.
const uint8_t spiSpeed = SPI_CLOCK_DIV8;

If I define a pin with the STM32F103C8 naming scheme (PAxx/PBxx/PCxx, etc…), the display works, but, if i compile the sdinfo test sketch, I have a different numbering on the serial monitoring. For exemple, my PB13 has the pin number 33 shown on the serial monitor.

I don’t know what else to do now…

stevestrong Mon Jun 13, 2016 11:18 am

It looks like issues are observed with SdFat together with ILI9341 lib.
Shouldn’t one start a separate thread for this?
I mean, here I would discuss issues related only to SdFat. Wouldn’t make sense?

VadimEL Sun Aug 21, 2016 5:00 pm

how to replace one char in position ???

I have 70Kb file f.txt, like
1234567890qwertyiopasdfghjkl

martinayotte Sun Aug 21, 2016 6:56 pm

file.seekSet(10);

VadimEL Sun Aug 21, 2016 8:01 pm

martinayotte

ok, thanks work fine, then I change to 1 ” #define SD_SPI_CONFIGURATION 1″
I can write to file and SeekSet is work

but led PB1 is lights, and all work fine….

pinMode(SD_CS, OUTPUT);
if (!SD.begin(SD_CS))
{
Serial.println("initialization failed!");

pinMode(PB1, OUTPUT);
digitalWrite(PB1, HIGH); // turn the LED on (HIGH is the voltage level)

return;
}
Serial.println("initialization done.");

martinayotte Mon Aug 22, 2016 1:03 pm

What do you mean ?
Do you mean it was working with SD_SPI_CONFIGURATION to zero and stop working when 1 ?

VadimEL Tue Aug 23, 2016 12:02 pm

#include <SD.h> has no seekSet function, and #include <SdFat.h> has not working with ” #define SD_SPI_CONFIGURATION 1″
I change to ” #define SD_SPI_CONFIGURATION 1″ and it work.

martinayotte Wed Aug 24, 2016 12:28 pm

First, the SD library and SdFat library are not the same.
In the first one, the function is simply named seek(), while in the second it is named seekSet().

Of course, for SdFat, you need SD_SPI_CONFIGURATION=1 since it will use the standard SPI.

Pito Wed Sep 14, 2016 11:06 am

FYI – the latest SdFat-beta (MMini, 36MHz SPI) and the “bench” with 512bytes buffer size and new SdFatEX class:
Type any character to start
FreeStack: 14444
Type is FAT32
Card size: 8.03 GB (GB = 1E9 bytes)

Manufacturer ID: 0X1B
OEM ID: SM
Product: 00000
Version: 1.0
Serial number: 0X8D172C54
Manufacturing date: 1/2015

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
3528.36,22171,135,143
3654.74,7632,135,138

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
3280.63,1512,154,154
3282.78,1429,154,154

Done
Type any character to start

Sign in to reply