Hi Jop.

The CardInfo example is just for arduino boards. If you try to compile for an Arduino board, the example works fine.

To test the card with the ESP32, you need to choose an example from the SD(esp32) library. Test the SD_Test sketch. Can you test that example?

It is working fine for me.

Which version of Arduino IDE are you using? I’m using 1.8.7.

Regarding the LoRa project, which board are you using? A board with built-in LoRa or a DOIT board + a LoRa module.

Are you using our exact same sketch or have you made some changes?

Can you send me some links to the “corrupted library discussions” so that I can better understand the problem and if it applies to the ESP32.

Regards,
Sara

Hello Sara,

Thanks again for you adequate answers; feels as really supporting.

First your points and questions:

You are right, CardInfo compiles for Arduino. How this works is not quite clear to me. Does internal source code for Arduino expose this ‘missing’ object Sd2Card? (I’m a newbie in this area).

Yes, I did already test that example and works fine for ESP32 and my SD-device.

My Arduino IDE version is 1.8.9.

I have both boards; the DOIT and with buit-in LoRa (Wemos TTGO LoRa ESP32 but for 868 Mhz).

And I can compile my sketch (based on the one of Rui) for both boards.

But now I’m talking with you about the DOIT where I installed the SDcard Module. So here my problems (did not test for the LoRa board).

Yes I made changes to the sketch, as for translating prompts to Dutch; later on I used for the routines where is written above “DON”T MODIFY THIS FUNCTION” exactly the code from your example. BTW, for what reason this warning is?

But your question brought me to “o, let us try the RNT-example as pure as it can be”. I changed just a few necessary things and I added DumpFile from SD example. I insert this code her below this post; where I made changes I added “@sara” with my remark”.

And now the remarkable thing: also this code has same problems with handling SD.

It is NOT appending new LoRa-messages to the log. Once in a while it does, but far too little.

I added later routine DumpFile() that can be triggered by a page refresh of the exposed HTML.

Here a screenshot of the resulting Serial output:

As you see, Sara, the logfile \data.txt doesn’t grow by appending the received Lora-messages, BUT routine DumpFile() (copied from SD-example sketch Dumpfile) is also giving weird output. And I’ve limited output as you can see in the code. It seems there is no EOF-marker.

Within sketch LoRa-receiverRUI the code Dumpfile() from SD Library works bad.

But if I do a DumpFile with the SD-example sketch Dumpfile I got this output (no weird characters). But still you see far too little appended LoRa-messages. The size is indeed 264 bytes.

Your last question, where I googled rumors about corrupted library:

• https://forums.adafruit.com/viewtopic.php?f=22&t=72289
• https://forum.arduino.cc/index.php?topic=440957.0
• https://forum.arduino.cc/index.php?topic=270065.0   in German, but you could switch on Google translator.

You can also google on error “‘Sd2Card’ does not name a type” in combination with Arduino or ESP32

And here a last remark of me: if you see description SD Library at Arduino site. Did you notice this bold remark?

Note that even if you don’t use the hardware SS pin, it must be left as an output or the SD library won’t work.

I tried this, but no improvement.

Thnx for you interest.

Friendly greetings,

Jop

code as used above: link to code

/*********
Rui Santos
Complete project details at http://randomnerdtutorials.com
*********/
 
// Import libraries
#include <WiFi.h>
#include <SPI.h>
#include <LoRa.h>

 

// Libraries to get time from NTP Server
#include <NTPClient.h>
#include <WiFiUdp.h>

 

// Libraries for SD card
#include "FS.h"
#include "SD.h"

 

// Replace with your network credentials
const char* ssid = "AGNIET";
const char* password = "Pelleboer";

 

// Define NTP Client to get time
WiFiUDP ntpUDP;
NTPClient timeClient(ntpUDP);

 

// Variables to save date and time
String formattedDate;
String dayStamp;
String timeStamp;

 

// LoRa Module pin definition
// define the pins used by the LoRa transceiver module
#define ss 5
#define rst 14
#define dio0 2

 

// Initialize variables to get and save LoRa data
int rssi;
String loRaMessage;
String temperature;
String soilMoisture;
String batteryLevel;
String readingID;

 

// Define CS pin for the SD card module
#define SD_CS 4 // 15 @Sara: changed this to my hardware setup as is now (can set it back to 15 if you like)

 

// Set web server port number to 80
WiFiServer server(80);

 

// Variable to store the HTTP request
String header;

 

void setup() {
// Initialize Serial Monitor
Serial.begin(115200);

 

// Initialize LoRa
//replace the LoRa.begin(---E-) argument with your location's frequency
//note: the frequency should match the sender's frequency
//433E6 for Asia
//866E6 for Europe
//915E6 for North America
LoRa.setPins(ss, rst, dio0);
while (!LoRa.begin(868E6)) { // @Sara: changed to 868 as my sender is using this, and it seems to me that this is the official frequency for Europe
Serial.println(".");
delay(500);
}
// Change sync word (0xF3) to match the sender
// The sync word assures you don't get LoRa messages from other LoRa transceivers
// ranges from 0-0xFF
LoRa.setSyncWord(0xF3);
Serial.println("LoRa Initializing OK!");

 

// Connect to Wi-Fi network with SSID and password
Serial.print("Connecting to ");
Serial.println(ssid);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
// Print local IP address and start web server
Serial.println("");
Serial.println("WiFi connected.");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
server.begin();

 

// Initialize a NTPClient to get time
timeClient.begin();
// Set offset time in seconds to adjust for your timezone, for example:
// GMT +1 = 3600
// GMT +8 = 28800
// GMT -1 = -3600
// GMT 0 = 0
timeClient.setTimeOffset(3600);

 

// Initialize SD card
SD.begin(SD_CS);
if(!SD.begin(SD_CS)) {
Serial.println("Card Mount Failed");
return;
}
uint8_t cardType = SD.cardType();
if(cardType == CARD_NONE) {
Serial.println("No SD card attached");
return;
}
Serial.println("Initializing SD card...");
if (!SD.begin(SD_CS)) {
Serial.println("ERROR - SD card initialization failed!");
return; // init failed
}

 

// If the data.txt file doesn't exist
// Create a file on the SD card and write the data labels
File file = SD.open("/data.txt");
if(!file) {
Serial.println("File doens't exist");
Serial.println("Creating file...");
writeFile(SD, "/data.txt", "Reading ID, Date, Hour, Temperature, Soil Moisture (0-4095), RSSI, Battery Level(0-100)\r\n");
}
else {
Serial.println("File already exists");
}
file.close();
}

 

void loop() {
// Check if there are LoRa packets available
int packetSize = LoRa.parsePacket();
if (packetSize) {
getLoRaData();
getTimeStamp();
logSDCard();
}
WiFiClient client = server.available(); // Listen for incoming clients

 

if (client) { // If a new client connects,
Serial.println("New Client."); // print a message out in the serial port
String currentLine = ""; // make a String to hold incoming data from the client
while (client.connected()) { // loop while the client's connected
if (client.available()) { // if there's bytes to read from the client,
char c = client.read(); // read a byte, then
Serial.write(c); // print it out the serial monitor
header += c;
if (c == '\n') { // if the byte is a newline character
// if the current line is blank, you got two newline characters in a row.
// that's the end of the client HTTP request, so send a response:
if (currentLine.length() == 0) {
// HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK)
// and a content-type so the client knows what's coming, then a blank line:
client.println("HTTP/1.1 200 OK");
client.println("Content-type:text/html");
client.println("Connection: close");
client.println();

 

// Display the HTML web page
client.println("<!DOCTYPE html><html>");
client.println("<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">");
client.println("<link rel=\"icon\" href=\"data:,\">");
// CSS to style the table
client.println("<style>body { text-align: center; font-family: \"Trebuchet MS\", Arial;}");
client.println("table { border-collapse: collapse; width:35%; margin-left:auto; margin-right:auto; }");
client.println("th { padding: 12px; background-color: #0043af; color: white; }");
client.println("tr { border: 1px solid #ddd; padding: 12px; }");
client.println("tr:hover { background-color: #bcbcbc; }");
client.println("td { border: none; padding: 12px; }");
client.println(".sensor { color:white; font-weight: bold; background-color: #bcbcbc; padding: 1px; }");

 

// Web Page Heading
client.println("</style></head><body><h1>Soil Monitoring with LoRa </h1>");
client.println("<h3>Last update: " + timeStamp + "</h3>");
client.println("<table><tr><th>MEASUREMENT</th><th>VALUE</th></tr>");
client.println("<tr><td>Temperature</td><td><span class=\"sensor\">");
client.println(temperature);
client.println(" *C</span></td></tr>");
// Uncomment the next line to change to the *F symbol
//client.println(" *F</span></td></tr>");
client.println("<tr><td>Soil moisture (0-4095)</td><td><span class=\"sensor\">");
client.println(soilMoisture);
client.println("</span></td></tr>");
client.println("<tr><td>Battery level</td><td><span class=\"sensor\">");
client.println(batteryLevel);
client.println(" %</span></td></tr>");
client.println("<p>LoRa RSSI: " + String(rssi) + "</p>");
client.println("</body></html>");

 

// The HTTP response ends with another blank line
client.println();
// Break out of the while loop
break;
} else { // if you got a newline, then clear currentLine
currentLine = "";
}
} else if (c != '\r') { // if you got anything else but a carriage return character,
currentLine += c; // add it to the end of the currentLine
}
}
}
// Clear the header variable
header = "";
// Close the connection
client.stop();
Serial.println("Client disconnected.");
Serial.println("");

 

dumpFile(); // @Sara: every client originating page-refresh is generating a dump logfile to serial output

 

}
}

 

// Read LoRa packet and get the sensor readings
void getLoRaData() {
Serial.print("Lora packet received: ");
// Read packet
while (LoRa.available()) {
String LoRaData = LoRa.readString();
// LoRaData format: readingID/temperature&soilMoisture#batterylevel
// String example: 1/27.43&654#95.34
Serial.print(LoRaData);

 

// Get readingID, temperature and soil moisture
int pos1 = LoRaData.indexOf('/');
int pos2 = LoRaData.indexOf('&');
int pos3 = LoRaData.indexOf('#');
readingID = LoRaData.substring(0, pos1);
temperature = LoRaData.substring(pos1 +1, pos2);
soilMoisture = LoRaData.substring(pos2+1, pos3);
batteryLevel = LoRaData.substring(pos3+1, LoRaData.length());
}
// Get RSSI
rssi = LoRa.packetRssi();
Serial.print(" with RSSI ");
Serial.println(rssi);
}

 

// Function to get date and time from NTPClient
void getTimeStamp() {
while(!timeClient.update()) {
timeClient.forceUpdate();
}
// The formattedDate comes with the following format:
// 2018-05-28T16:00:13Z
// We need to extract date and time
formattedDate = timeClient.getFormattedDate();
Serial.println(formattedDate);

 

// Extract date
int splitT = formattedDate.indexOf("T");
dayStamp = formattedDate.substring(0, splitT);
Serial.println(dayStamp);
// Extract time
timeStamp = formattedDate.substring(splitT+1, formattedDate.length()-1);
Serial.println(timeStamp);
}

 

// Write the sensor readings on the SD card
void logSDCard() {
loRaMessage = String(readingID) + "," + String(dayStamp) + "," + String(timeStamp) + "," +
String(temperature) + "," + String(soilMoisture) + "," + String(rssi) + "," + String(batteryLevel) + "\r\n";
appendFile(SD, "/data.txt", loRaMessage.c_str());
}

 

// Write to the SD card (DON'T MODIFY THIS FUNCTION)
void writeFile(fs::FS &fs, const char * path, const char * message) {
Serial.printf("Writing file: %s\n", path);

 

File file = fs.open(path, FILE_WRITE);
if(!file) {
Serial.println("Failed to open file for writing");
return;
}
if(file.print(message)) {
Serial.println("File written");
} else {
Serial.println("Write failed");
}
file.close();
}

 

// Append data to the SD card (DON'T MODIFY THIS FUNCTION)
void appendFile(fs::FS &fs, const char * path, const char * message) {
Serial.printf("Appending to file: %s\n", path);

 

File file = fs.open(path, FILE_APPEND);
if(!file) {
Serial.println("Failed to open file for appending");
return;
}
if(file.print(message)) {
Serial.println("Message appended");
} else {
Serial.println("Append failed");
}
file.close();
}

 

// @Sara: added this below, "stolen' from SD example DumpFile

 

#define logfileSD "/data.txt"

 

void dumpFile() {
// open the file. note that only one file can be open at a time,
// so you have to close this one before opening another.
File dataFile = SD.open(logfileSD);

 

// if the file is available, write to it:
if (dataFile) {
Serial.print(dataFile.name());
Serial.print("\t\tsize of file: ");
Serial.println(dataFile.size(), DEC);

 

int count = 0; // @Sara, if I don't limit here this output I got an endless row of rubish in my serial ouput (and system is in a kind of hang-up)
while (dataFile.available() && count < 2000) {
Serial.write(dataFile.read());
//Serial.print(dataFile.read());
count++;
}
Serial.println();

 

dataFile.close();
}
// if the file isn't open, pop up an error:
else {
Serial.println("error opening file '" + String(logfileSD) + "'");
}
}

Hi again.

Even though you get garbage when you call the dumpFile() function, have you checked the microSD card content and see if it is writing something?

Which SD card module are you using?

I think that the issue might be due to be using the same SPI pins both for LoRa and the SD card. (however, it worked well as it is when we tested it).

The “Don’t modify this function” message refers to the function that writes to the SD card. If people modify that function the wrong way, they might not be able to write to the sd card. It is just to prevent errors.

Regards,
Sara

—-WRITTEN BY Jop via email—

Hi Sara,

It took a lot of time but after long searching and changing code and debugging I found that the combination of WiFi and this SD Card routines can NOT stay together in one sketch.

If I disable completely all code in this sketch that has to do with WiFi, (connecting to local network and setting up a webserver), then no problems with SD Card.

Without handling WiFi, appending lines is going well and dumpFile() is showing no garbage but just the content as it is.

As I wrote before also your original LoRa-Receiver sketch (see listing I gave in my previous post) is suffering from this SD Card problems.

Your questions: no, (with WiFi implemented) he is not appending lines to \data.txt but the existing content is also kept as it is. dumpFile() is displaying garbage but actually there is no garbage in the file.

I’m using this SD Card module: banggood.com/3Pcs-Micro-SD-TF-Card-Memory-Shield-Module-SPI-Micro-SD-Adapter-For-Arduino-p-1003057.html

What can we do now we know that WiFi and SD Card have a conflict?

Kind regards,

Jop

Hi again.

Our sketch worked well with our setup. So, I’m not sure what might be wrong in your case.

I don’t know if it can be something to do with your SD card module. We’re using a slightly different one. We’re using this: https://makeradvisor.com/wp-content/uploads/2017/11/micro-sd-card-module-img.jpg

I also think it might be an issue because we’re using the same SPI pins for LoRa and the SD card. Maybe you can try using another SPI instance. See this discussion: https://github.com/espressif/arduino-esp32/issues/1219 and see if you can try implementing the SD card on other SPI pins.

If it still doesn’t work. You can replace the microSD card with publishing sensor readings on Google sheets for example, as you did in your other project to log your data.

I really want to help you, but I’m out of ideas on how to solve this problem :/

Please check the SPI pins, I think that might be the problem. Also, try changing the SD card CS pin to another pin (try GPIO34).

Regards,

Sara

Hi Sara,

I tried your sketch (with small adaptations, you can see them at @sara) but this has same problems as my sketch.
Yes, I have a different SD card module, mainly because your one on MakerAdvisor was giving a dead link.
Now I have 2 different ones, this one (works as long as I disable WiFi) and this catalex (see my post) that doesn’t even mount (did not check this with disabling WiFi).

The discussion, you gave the link, I read with attention. I think suggestion in the last post (of mr. fa1ke5) in this discussion is the most elegant as not having to change the libraries.
It uses these pins: 
spiSD.begin(14,2,15,13);//SCK,MISO,MOSI,ss // инициализируем HSPI1
and SD_CS = 13
  (o, this Cyrillic characters are Russian and meaning “Initialization”)
  
But it uses pin also pin 14 and this is used in your RNT-project for the RESET of the RFM95 module.
Can we change this to another pin? And to which pin, you would suggest?

Besides that: why this mr. fa1ke5 is not using the HSPI-pins as given in the 30-pins-layout? Btw, can most pins act as SPI-pins?
My suggestion for this spiSD.begin(14,2,15,13) would be: 
spiSD.begin(14,12,13,15);//SCK,MISO,MOSI,ss  (Jop: ss is same as or a naming for CS?)
with SD_CS=15
I’m talking about the DOIT ESP32 (not the LoRa-equipped ESP32, brings more difficulties)
Do you agree that is better, if we can change pin 14 for the LoRa RFM95 to …?

Yes, I did already publishing GSheets, but for the future I’m willing to know more about these SPI-matters. I read on internet more rumors that SPI is conflicting with WiFi. Or conflicts between to SPI-driven modules.
Besides that – this LoRa-receiver unit has sometimes hangups and I have to reset. Luckily the solar driven sender stays working. I suspect this SPI-problems as causing crashes (even as I disabled the SD-card software in the sketch).

I tried your suggestion SD_CS = 34, but that doesn’t work at all – even not mounting of SD-card. Checked this also with the example sketch SD_Test (for ESP32). Both pin4 or pin15 work good, is my experience.

Yes, I really feel you want to help me. Thanks for that. 🙂

Have a nice day.

        Jop

Hi again.

I’m sorry for the broken links. I’ve fixed the links on Maker Advisor this morning.

You can change the pins of the LoRa module,for example, you can change the RST pin to GPIO 34, because GPIO 14 is a HSPI pin by default.

I agree with the pins you suggested for the microSD card. Those are the HSPI pins.

However, you don’t necessarily need to use the HSPI pins. You can set almost any pins to act as SPI by setting them on software.

There are pins more suitable than others. For example, if you want to pick different pins I would go with the “safe to use” ones highlighted in green in our table in this link: https://randomnerdtutorials.com/esp32-pinout-reference-gpios/

Regards,
Sara

Then, Sara, pin 34 for SD_CS was not a good idea, I see in your table in the last link, as it is not suitable as ouput. ;). Clarifying possibly my bad results voor mounting SD. Or am I completely wrong an SD_CS has only input role?

O and here again, for RST for the RFM95 – I guess the software and the CPU should give a RESET command to the RFM95?

Regards,

     Jop

Hi Jop.
You are right. It is an input only. I’m sorry. I was looking at an old pinout table that I have here.
So, don’t use GPIO 34. Choose another that is suitable for output.
I’m sorry for my mistake.
 

O no excuses needed, Sara, I make every day mistakes, we are humans.
And I learned from it, that is to be aware of Input/ouput characteristic of the pins
I tried the HSPI but terrible results – I cannot even upload the sketch to the ESP32, only when I disconnect the VCC of the SD card module. And when (after uploading the sketch) I connect this VCC again and do an RESET,
the ESP seems to reboot constantly. I see these prompts continuously in serial window.

rst:0x10 (RTCWDT_RTC_RESET),boot:0x33 (SPI_FAST_FLASH_BOOT)
flash read err, 1000
ets_main.c 371
ets Jun 8 2016 00:22:57
Also pin-configuration out of this forum link you gave doesn’t work.

So I give it up. Conclusion: Wifi and SPI for this SD Card have to serious conflicts.

Thanks for all your support,

      Jop

Hi Jop.
I’m sorry for the issue. I really wanted to make it work, but at this moment I’m out of ideas on how to solve it :/
Meanwhile, if you need help with other subjects, feel free to ask.
Regards,
Sara

Hi Sara,

Just another question about LoRa-receiver ESP32 – I’m talking about the DOIT board.

Sometime (almost once a day) this is crashing or in coming a hangup or something like that. (I don’t use the SDcard anymore; disconnected0.
I have to reset it and then it works for another day or halfday. 

Do you have any idee what the reason could be?

I hardly can image it is the software.

Th sender unit – standing outside my house an running on solar and with the sleep mode – is working perfect and no crashes.

Regards,
   Jop

Hi Jop.
The receiver (the one running the web server) crashes in what way? Does it disconnect from wi-fi?
We can add something that checks if it is connected to wi-fi and if it isn’t, try to reconnect.

Hi Sara,

Maybe that’s a good suggestion (your last line). Do you have code for that, ready to insert?

For me it is not clear if it is only disconnected from WiFi or completely crashed. Anyway, I cannot refresh the serverpage and further also GSheets is not updated, so it could be the WiFi-connection.

Almost the same sketch is running on the TTGO Lora-equipped ESP32 and there no problems – this is the same sketch but with compiler directives tuned for the TTGO. So for that reason I suspected the hardware, you see.

Thnx and regards,

      Jop

Yes.

You can use the following snippet to check if it is connected to wi-if. If it isn’t reset the board via software:

if (WiFi.status() != WL_CONNECTED){
  Serial.println("Resetting");
  ESP.restart();
}

Look at your code and see the most appropriate section to include this snippet.

You can also follow this discussion about automatically reconnect to Wi-Fi:

https://rntlab.com/question/wifi-connection-drops-auto-reconnect/

I hope this helps.

Regards,
Sara

Hello Sara,

Your tip about losing WiFi-connection I implemented. I read the discussion on the link you gave. So I did not restart the ESP but let him reestablish the WiFi-connection according code in this link. Tested this by switching off my WiFi-router for a while.

But this was not the solution for my problem. It stuck me that the problem was mainly with the VDOIT ESP32 and not with the TTGO ESP32. 
This makes me thinking again that the external LoRa-transceiver, as it is controlled by SPI from the ESP32, is not complete stable with WiFi working (see our conversation about the SDCard-module not working with WiFi in use).
The TTGO has an internal LoRa-module (also SPI controlled) and probably this integrated configuration works better.

I decided to keep administration in my code if receiving LoRa-packages is going on. In my case every 10 minutes such a package should be received (under good LoRa-transmit conditions). If more as 3 packages on a row are missing the code tries to restart with ESP.restart() (learned from your tip).

BUT…..  as it turns out, this doesn’t be a solution. I think because the ESP32-code is completely crashing now and then and in a long time, so not able to perform such restart(). Do you think my assumption is right?

I have to say that during my 2,5 week holiday (short ago) I have let run a long  ‘unwatched’ test. I tested these systems with one LoRa-transmitter in my garden and 2 receiving LoRa-receivers, within my house,  catching the packages and writing them to 2 different Google Sheets. And this has been working good, except for missing now and then packages. But no crashing VDOIT, now.
I have to say that distance between receiving VDOIT and transmitter was only a few meter ( with glass between it).
So it is difficult to trap this problem as I encountered now and then.
I have to say that the LoRa-transmitter side, powered by only solar panels, is working very stable and good. One should expect that this would be the “weak link in the chain”. But not. It has been doing its job for almost 3 weeks continuously (missing packages due to the receivers – also for that reason I used 2 receiving LoRa-stations)

About “missing packages” I will come with suggestions later in a new post on this forum.

Thanx for all your friendly an patient help so far.

Kind regards,

    Jop

Hi Jop.

Thank you so much for sharing your results with us.

I don’t know why the ESP32 DOIT crashes. As you’ve said, it may be something related with the SPI, because the microSD card also didn’t work well. Or it can be some GPIO that you’re using that should be changed, but it is hard to tell.

As I’ve told you previously, this project worked successfully on our side (otherwise we wouldn’t include it in the course). But I’m out of ideas on what may be causing the issue :/

I think missing packets should be normal once in a while since we’re using cheap LoRa transceiver modules.
Let me know if you need further help on something else.

Regards,

Sara