I am trying to embed the OTA updater into a sketch that already uses a webserver for read out and control via a web page.
I have set up a test program to try and get this to work. The idea is start it in webserver mode and use the web page to provide an option to switch back into OTA mode. To do this I am stopping one server server1.stop and starting the next with server2.begin.
However it is not working.
Each mode works independently. ie if I start in OTA mode, that works and webserver mode also works fine. The issue seems to be switching from one to the other. I am probably doing something simple wrong. Help!
I attach the test code below:
#include "Wire.h" #include "RTClib.h" // Load Wi-Fi library #include <WiFiMulti.h> #include <WiFi.h> #include <WiFiClient.h> #include <WebServer.h> #include <ESPmDNS.h> #include <Update.h> //WiFiMulti WiFiMulti; WiFiServer server1(80); WebServer server2(80); boolean upload_mode = 0; //use upload mode boolean switch_mode = 0; const int buttonPin = 32; // the number of the pushbutton pin int buttonState = HIGH; // variable for reading the pushbutton status int indicator = 2; //little blue light to signify something on the board //int lamp = 5; //output for lamp relay //int door = 4; //output for door relay const char* host = "esp32"; const char* ssid = "SSID"; // Your SSID (Name of your WiFi) const char* password = "PASSWORD"; //Your Wifi password // Variable to store the HTTP request String header; /* * Login page */ const char* loginIndex = "<form name='loginForm'>" "<table width='20%' bgcolor='A09F9F' align='center'>" "<tr>" "<td colspan=2>" "<center><font size=4><b>ESP32 Login Page</b></font></center>" "<br>" "</td>" "<br>" "<br>" "</tr>" "<td>Username:</td>" "<td><input type='text' size=25 name='userid'><br></td>" "</tr>" "<br>" "<br>" "<tr>" "<td>Password:</td>" "<td><input type='Password' size=25 name='pwd'><br></td>" "<br>" "<br>" "</tr>" "<tr>" "<td><input type='submit' onclick='check(this.form)' value='Login'></td>" "</tr>" "</table>" "</form>" "<script>" "function check(form)" "{" "if(form.userid.value=='admin' && form.pwd.value=='admin')" "{" "window.open('/serverIndex')" "}" "else" "{" " alert('Error Password or Username')/*displays error message*/" "}" "}" "</script>"; /* * Server Index Page */ const char* serverIndex = "<script src='https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js'></script>" "<form method='POST' action='#' enctype='multipart/form-data' id='upload_form'>" "<input type='file' name='update'>" "<input type='submit' value='Update'>" "</form>" "<div id='prg'>progress: 0%</div>" "<script>" "$('form').submit(function(e){" "e.preventDefault();" "var form = $('#upload_form')[0];" "var data = new FormData(form);" " $.ajax({" "url: '/update'," "type: 'POST'," "data: data," "contentType: false," "processData:false," "xhr: function() {" "var xhr = new window.XMLHttpRequest();" "xhr.upload.addEventListener('progress', function(evt) {" "if (evt.lengthComputable) {" "var per = evt.loaded / evt.total;" "$('#prg').html('progress: ' + Math.round(per*100) + '%');" "}" "}, false);" "return xhr;" "}," "success:function(d, s) {" "console.log('success!')" "}," "error: function (a, b, c) {" "}" "});" "});" "</script>"; //-------------------------------------------------------- void setup() { //Serial Port begin Serial.begin (9600); pinMode(indicator, OUTPUT); pinMode(buttonPin, INPUT); Connect_to_Wifi(); //set up wifi server if(upload_mode) connect_upload_server(); //set up upload server if(upload_mode) server2.begin(); else server1.begin(); } void loop() { // read the state of the pushbutton value: buttonState = digitalRead(buttonPin); // check if the pushbutton is pressed. If it is, the buttonState is HIGH: //if (buttonState == LOW) //state goes low when pressed if(switch_mode == 1) { Serial.println("upload button pressed"); upload_mode = 1; switch_mode = 0; server1.stop(); delay(1000); //WiFi.disconnect(); //Connect_to_Wifi(); server2.begin(); Serial.println("upload page active"); delay(1000); connect_upload_server(); } if(!upload_mode) { check_server(); } if(upload_mode) { server2.handleClient(); delay(1); } } //------------------------------------ void connect_upload_server() { //use mdns for host name resolution if (!MDNS.begin(host)) { //http://esp32.local Serial.println("Error setting up MDNS responder!"); while (1) { delay(1000); } } Serial.println("mDNS responder started"); //return index page which is stored in serverIndex server2.on("/", HTTP_GET, []() { server2.sendHeader("Connection", "close"); server2.send(200, "text/html", loginIndex); }); server2.on("/serverIndex", HTTP_GET, []() { server2.sendHeader("Connection", "close"); server2.send(200, "text/html", serverIndex); }); //handling uploading firmware file server2.on("/update", HTTP_POST, []() { server2.sendHeader("Connection", "close"); server2.send(200, "text/plain", (Update.hasError()) ? "FAIL" : "OK"); ESP.restart(); }, []() { HTTPUpload& upload = server2.upload(); if (upload.status == UPLOAD_FILE_START) { Serial.printf("Update: %s\n", upload.filename.c_str()); if (!Update.begin(UPDATE_SIZE_UNKNOWN)) { //start with max available size Update.printError(Serial); } } else if (upload.status == UPLOAD_FILE_WRITE) { // flashing firmware to ESP if (Update.write(upload.buf, upload.currentSize) != upload.currentSize) { Update.printError(Serial); } } else if (upload.status == UPLOAD_FILE_END) { if (Update.end(true)) { //true to set the size to the current progress Serial.printf("Update Success: %u\nRebooting...\n", upload.totalSize); } else { Update.printError(Serial); } } }); } //------------------------------------ void Connect_to_Wifi() { // We start by connecting to a WiFi network //WiFiMulti.addAP(ssid, password); // Connect to WiFi network WiFi.begin(ssid, password); Serial.println(); Serial.println(); Serial.print("Waiting for WiFi... "); while (WiFi.status() != WL_CONNECTED) { delay(500); Serial.print("."); } Serial.println(""); Serial.println("WiFi connected"); Serial.println("IP address: "); Serial.println(WiFi.localIP()); digitalWrite(indicator, HIGH); } //------------------------------------ void check_server() //check web server and update { WiFiClient client = server1.available(); // Listen for incoming clients if (client) { // If a new client connects, Serial.println("New Client."); // print a message out in the serial port String currentLine = ""; // make a String to hold incoming data from the client while (client.connected()) { // loop while the client's connected if (client.available()) { // if there's bytes to read from the client, char c = client.read(); // read a byte, then Serial.write(c); // print it out the serial monitor header += c; if (c == '\n') { // if the byte is a newline character // if the current line is blank, you got two newline characters in a row. // that's the end of the client HTTP request, so send a response: if (currentLine.length() == 0) { // HTTP headers always start with a response code (e.g. HTTP/1.1 200 OK) // and a content-type so the client knows what's coming, then a blank line: client.println("HTTP/1.1 200 OK"); client.println("Content-type:text/html"); client.println("Connection: close"); client.println(); if (header.indexOf("GET /Upload/on") >= 0) { switch_mode = 1; } else if (header.indexOf("GET /Upload/off") >= 0) { switch_mode = 0; } // Display the HTML web page client.println("<!DOCTYPE html><html>"); client.println("<head><meta name=\"viewport\" content=\"width=device-width, initial-scale=1\">"); client.println("<link rel=\"icon\" href=\"data:,\">"); // client.println("<meta http-equiv=\"refresh\" content=\"2\">"); // CSS to style the on/off buttons // Feel free to change the background-color and font-size attributes to fit your preferences client.println("<style>html { font-family: Helvetica; display: inline-block; margin: 0px auto; text-align: center;}"); client.println(".button { background-color: #4CAF50; border: none; color: white; padding: 16px 40px;"); client.println("text-decoration: none; font-size: 30px; margin: 2px; cursor: pointer;}"); client.println(".button2 {background-color: #555555;}</style></head>"); // Web Page Heading client.println("<body><h1>TEST PAGE</h1>"); //client.println("<p>Upload Mode " + String(switch_mode) + "</p>"); if(!upload_mode) client.println("<p><a href=\"/Upload/on\"><button class=\"button\">Upload on</button></a></p>"); else //client.println("<p><a href=\"/Upload/off\"><button class=\"button button2\">Upload off</button></a></p>"); client.println("</body></html>"); // The HTTP response ends with another blank line client.println(); // Break out of the while loop break; } else // if you got a newline, then clear currentLine currentLine = ""; } else if (c != '\r') // if you got anything else but a carriage return character, currentLine += c; // add it to the end of the currentLine } } // Clear the header variable header = ""; // Close the connection client.stop(); Serial.println("Client disconnected."); Serial.println(""); } }
Hi David,
Your code is not very long readable in this form…
It would be better to post it on GitHub Gist and send us the link here 🙂
Apologies. You can find the code here:
https://gist.github.com/davewhughes/82172a33c2e39d03d5e9c4866ccb2477
Thank you… it’s much more readable that way!
I haven’t played with OTA updates yet (I’m discovering that it’s possible with your post by the way!), but intuitively, I don’t really understand the interest of handling that with 2 servers (?) You can do everything with the same one, can’t you?
And in case you want to have several servers, I guess it’s not possible to attach them to the same port. You use port 80 for both, and I guess the pbm has to be linked to that.
I might be wrong (I’m still a beginner), but I have to experiment by myself. I already have to dig into this OTA thing.
Yes it probably does make sense to use 1 server. I wondered about that and tried that originally. However if I used 1 server with either just using WiFiserver or WebServer alone, I got errors in one part of the code or the other depending on whether it was the OTA code or the web page code. I don’t understand enough about how these commands work to know what I am really doing wrong. Probably very basic errors, but I am of the copy/adapt/try it and ‘learn if it works’ school of coding!
So for example, if I just use WiFiServer, I get “class WifiServer has no member named handleClient” and if I just use WebServer, I get class “WebServer has no member named available error”
Okay, I’m going to try to dive into this and figure out what the right thing to do is. You’re just going to have to be a little patient so that I have time to study the programming aspects of a web server capable of handling an OTA update.
I work all day long, so I can only devote myself to this in the evening.
I’m a beginner like you and I’m experimenting as much as I can, but it all takes time.
Maybe Sara or Rui can shed some light before me!
But the subject interests me, so I’m going to put on hold what I’ve been doing so far to better study your pbm.
I hope I won’t be too long :-/
You may have figured it out on your own by then! 🙂
Thanks for your help and perseverance. I doubt I will have figured it out on my own. I have been working at it for some time already and have reached the limit of my knowledge/expertise without outside help. It would be really useful to update projects by OTA, but it is shame to limit it to those that are not using wifi. Most of my projects do. So having webserver and OTA play nicely together is incredibly useful.
Ah… okay… I think I understand your underlying problem.
You actually want to be able to perform an OTA update on a project that already uses ESP32’s WiFi capabilities on its own… Alright!
But the downside, and I guess you get the point, is that you’ll have to integrate the necessary OTA update routines into ALL your projects!
Anyway, it’s still interesting and useful, you’re right.
By searching a little bit on Sara & Rui’s website (I haven’t explored everything yet!), I found this tutorial :
https://randomnerdtutorials.com/esp32-over-the-air-ota-programming/
I guess you’ve already studied it, haven’t you?
It’s just to understand at what stage of knowledge you are at and to make sure you haven’t missed the obvious.
Just to make things clearer…
You want to perform an OTA update through a web server (so an HTTP server, which usually listens on port 80) on an ESP32 which would already be running a socket server (a TCP server, but not necessarily an HTTP server), so totally independent from the HTTP server.
In this case, it seems clear to me that the two servers must be distinguished, which must be executed separately, and therefore listen on different ports! Your socket server must not listen on port 80, which would then be reserved for the web server (for the OTA update).
Now, if the core of your main project also uses a web server (i.e. HTTP, which listens on port 80), then it is precisely this same server that will have to take care of the update. And you will have to plan a specific HTTP route (a specific path in the URL) whose function will be precisely to allow the update.
I don’t know if I’m clear enough?
Yes, I used Sara and Rui’s article initially to get started.
I don’t have a problem putting the OTA code into all projects.
What I am trying to do quite simple really. When the ESP32 boots up, it starts a web server, so that I can monitor values and send instructions to device. One of those instructions is to put it into OTA mode so I can upload a new version of the code. When it reboots it goes back into websserver mode.
Well that is the plan anyway.
Okay, so it’s the simplest case!
This corresponds to the second solution I mentioned in my previous message.
The heart of your project is a web server that listens on port 80 and does everything you ask it to do: display values and respond to actions you tell it to do (through HTML buttons for example).
In this case, the HTTP route is the default: /
In the tutorial, this route displays the password-protected login panel to access the upload page of the new sketch. But in your case, you can simply replace this page with an interface that displays all the values you want to monitor, as well as a button that allows you to switch to the upload page of the new sketch for an update.
To place your server in update mode (without compromising its default behavior), you just need to provide an HTTP route that will present a new web interface in order to send it the compiled code of a new sketch so that it can update itself according to the OTA protocol.
In the tutorial, the corresponding HTTP route is precisely: /serverIndex
This page just presents a new web interface with a form that allows you to upload your new sketch.
By validating the form, your browser switches to a new page that will actually perform the transfer of the sketch and perform the update. This HTTP route is: /update
When the server receives the request corresponding to this route, it updates the sketch and restarts.
This tutorial is exactly what you’re trying to do!
If that’s not clear enough for you, I can code another example, which may be more in line with my idea of what you want to do.
But it’s already 11:00pm here (I live in Reunion Island, in the Indian Ocean, which is at GMT+4) and I have to get up early tomorrow. So I will try to prepare this for you tomorrow evening. I’ll try to do it as soon as possible, hoping not to be disturbed by other imperatives…
I am sort of understanding, but not sure how I would implement it to be honest, so an example would be really helpful. I understand C code well enough, but I am at a bit of a loss when it comes to html. I am learning slowly but do not yet know enough to implement this.
Thanks for all your help.
I am sort of understanding, but not sure how I would implement it to be honest, so an example would be really helpful. I understand C code well enough, but I am at a bit of a loss when it comes to html. I am learning slowly but do not yet know enough to implement this.
Thanks for all your help.
Hi David,
Just a quick note to tell you that I haven’t forgotten you! I had to learn a lot of new things to be able to help you properly. I finally found the time to code something for you today 🙂
Here’s a preview of two web applications I coded to interact with ESP32, and for which I just have to implement the OTA update. I’ll try to do this tomorrow.
I hope it’s pretty much what you wanted…
These applications work just as well with the ESP32 in access point mode as when it is connected to an ambient WiFi network (coming from a router).
A little more patience and you’ll soon know how to make it all come true.
I’ll drop everything on GitHub for you.
Stéphane
Hi Stephane, I was wondering if you have had any success in getting this to work?
Hi, David,
I have had little time this week to work on ESP32. But I haven’t forgotten about you! I had planned to write a mini-tutorial on what I did, but if you’re in a hurry, I might as well publish my code as is on GitHub, if you think you can handle it.
Hi David,
I was finally able to work on my project today, and I took the opportunity to make it as simple as possible for you to use it easily.
I still have some cleaning up to do in my code before publishing it on GitHub, but here’s what it looks like:
[EDIT]: Apparently, you can’t embed a YouTube video directly here… so here’s the demo URL:
ESP32 WebSockets & OTA Web Updates
I deliberately avoided an object-oriented design so as not to add complexity to your understanding of what is essential for you here.
The WiFi connection of the ESP32 can be made either on an infrastructure network or as an access point (it all depends on what you prefer).
I hope you’re not too impatient, unfortunately I’ve been very busy with other imperatives.
Well, I’m trying to hurry up and publish my code!…
Dear David,
Thank you for your patience… I’m sorry it took so long, but unfortunately I couldn’t devote all the time I wanted to this project.
Anyway, as I promised you, I prepared a small project to illustrate what you were trying to achieve. So I took the opportunity to publish it in a specific thread, so that everyone could benefit from it:
ESP32 Web Apps with WebSockets & OTA Web Updates
I hope this project lives up to your expectations!
Friendly,
Steph
Hi David,
I haven’t heard from you… did you have any problems with what I sent you? Or maybe you haven’t had time to look at it yet?
Don’t hesitate to ask me if you need anything.
I have had a look, thanks Stephan. I am not a very advanced programmer, and there is a lot of stuff in your example I don’t really understand, so I am having trouble translating what you have done into what I have done so far.
I suspect I only need to change a few lines in my code, but can’t really figure out which ones from the example you have done, which looks amazing, but there is so much in it that is alien to me.
I am sorry, it feels really ungrateful after all the effort you have put in, but it all looks so different and new.
Okay, don’t get discouraged!
We can take things one step at a time 😉
The code you initially posted is much more complicated than mine, I assure you! In what I proposed, I tried to structure things a little bit more, by isolating the different components of the application, so that you can more easily dissect them.
But we can take it all back step by step.
What I can offer you (within the limit of my availability unfortunately), is to design a small project (with a simple LED for example), and to build with you the whole project step by step so that you understand each component, until the web OTA update stage.
Before doing so, I would need you to confirm that you are comfortable with what is detailed in this project:
If you are uncomfortable with what is explained there, we will have to take things step by step there as well, before going any further.
Are you okay with that?
Ok. I am looking at the SPIFFS project now. There are quite a few new concepts here for me. I am absolutely not an html person, so I have just adapted previous code to achieve my ends. I will need to try and get my head around this. I appreciate my code was inefficient, but I did sort of understand it. I will let you know when I have figured out what is going on in the SPIFFS example you referenced. Again thanks for your patience.
Yeah… but if you want to make web interfaces to control your ESP32, I think it’s time to get into HTML, at least a little! If you don’t do that, there’s too many things you’ll misunderstand. And you won’t be autonomous.
The tutorial on the web server using SPIFFS should make things a lot easier for you to understand, since it explains how to host the user interface files on SPIFFS: the HTML file and the CSS file.
Then, the way of coding the main sketch in C becomes very light since it doesn’t include the hard coding of these interfaces anymore. It focuses mainly on the web server and the way it serves these resources to the client browser.
Totally appreciate that. I am learning new things and better way of doing so all good. I have looked at the example now and am happy with the way it is done and can follow the logic. So ready for the next step!
Hi, David,
I’m glad you’re more comfortable with these notions. To start our journey, I would like to know if you are interested in looking into the creation of the HTML/CSS web interfaces that will be sent to the browser to interact with ESP32. If this part doesn’t interest you more than that, I can create a simple interface and you’ll just have to copy the corresponding files on your side to start the project. It’s up to you.
I am preparing an online document so that you can follow the development of the project and implement its different steps.
However, please let me know if you still need a detailed look at each step. You may have already figured it all out by now 🙂
I am sorry. I have sort of given up trying to get my head around this.
There are too many new concepts for me to take in at once. What I was looking for was just a simple way of updating my code with OTA but also being able to run a v simple webserver to report back some info at the same time.
I am a real novice at this and can just about get by with cutting, pasting and adapting code. What I was really looking for was a tweak to my code to get it working rather than a complete rewrite with lots of new concepts to learn.
I am sorry. I feel I am being ungrateful, but I am just struggling to understand the code you sent over. It looks so different to mine, which I do understand. Just!
David
Hello David,
Don’t feel ungrateful! I fully understand that you have difficulty understanding the code and that it takes time to assimilate things. But I don’t think that just copying and pasting code and trying to partially adapt it to your needs is a good approach. Indeed, even in Rui and Sara’s tutorials, which remain quite simple in general, many notions can be addressed on one and the same project. And just mimicking what is presented won’t teach you anything in the long run.
I think it is better to break the project down into simpler sub-projects that focus on only one notion at a time. For example, start by tackling how to turn on a simple LED with a push button, trying to work on all the aspects of this problem: how to de-interference the reading of the signal coming from the button, how to eliminate the bouncing effect induced by the mechanical nature of the button and be sure to respond properly to the user’s interactions, etc.
You know that, in the context of a project that uses a web server, you may need to store files in Flash memory with SPIFFS… so take a closer look at how to use SPIFFS.
Do you need to implement a web server with ESPAsyncWebServer? Start by reading the documentation! Then, proceed step-by-step, starting by making sure you are able to initialize the server. Then be able to send a simple file from SPIFFS to the client web browser. Do you want to be able to inject some data into the file before sending it to the client? Focus on the possibilities that exist to do so. And don’t be in too much of a hurry to take the next step! First make sure you understand all the outlines of the current stage of your progress. There is no point in wanting to go faster than the music. It’s the key to independence. If you don’t make this effort, and if you limit yourself to mimicking what others do, without fully understanding it, you will never be autonomous, and you will find it very difficult to adapt an existing project to what you would like to add or modify for your own project.
I know that all this takes a lot of time. I was like you until a few months ago. Everything seemed so complicated. I wanted to do a lot of stuff, but it required so many basics to be able to put together all the bricks needed to do what I wanted to do that it seemed insurmountable. So I scaled down my ambition to tackle simpler projects, focusing on learning and mastering fewer problems. I spent a lot of time there. But with motivation and the desire to learn how to master what I was doing, I can see much more clearly today. And I feel perfectly autonomous on many subjects. Of course, I still have a lot to learn. And the more I learn, the more I realize that I know nothing. But I’m moving forward, step by step, trying to make sure that I’ve understood one step before moving on to the next. Sometimes I even go back to things I thought I had mastered to perfect them, because in the meantime, I’ve read somewhere new information that makes it possible to do things better. Learning makes us constantly question what we thought we had learned as we progressed. You have to remain humble and accept that you have to go backwards to move forward more easily or quickly afterwards.
I still have a long way to go, but I’m moving forward without wanting to skip any steps. And I don’t copy/paste. I prefer to understand what I’m doing.
In the solution I proposed to you, it seems to me that the code is much clearer and much more digestible than what you submitted in your first posts. I made sure to break things down so that you can better isolate the different gears that make the machine work. If you have difficulty reading it, I encourage you to take each module in isolation to make sure you understand it. Once you have gone through each module, the assembly will seem much simpler.
I didn’t necessarily want to encourage you to tinker with something just to make it work. When I have some time, I write detailed tutorials around a simple project, but which allows you to approach in depth 2 or 3 fundamental notions. Concerning the ESP32, I published a tutorial on sleep modes:
Experimenting with ESP32 sleep modes
Take a look at it, you never know? Maybe it’ll help you see some things more clearly.
I will soon publish a tutorial that should interest you more, on how to use WebSockets with ESP32. You’ll think it’s still a new notion to acquire. Yes it is, but it’s a notion that will be useful in all your web projects! And in particular in the project that brought us together here. If you’re interested, I’ll keep you up to date as soon as I finish it.
Thanks for the update. I look forward to your websockets tutorial and I promise I will go through it in detail and try and understand it all!
Hi, David,
I hope you’re okay. I’ve been very busy at work these past few weeks, so I hope you’ll excuse the delay in my reply…
I finished my tutorial on implementing the WebSocket protocol with ESP32.
You’ll find the link here:
[Full Tutorial] ESP32 Remote Control with WebSocket
I hope it will suit you 😉
Regards,
Steph