FSBrowser does not work about autoconnect

[Hieromon]

It contains two errors.

1. /edit returns FileNotFound

[E][WString.cpp:185] changeBuffer(): realloc failed! Buffer unchanged

This log is output by the ESP32 core. PageBuilder 1.3.0 is degrading and buffer allocation fails. This log indicates the same phenomenon. If you use PageBuilder 1.3.0, please you replace the PageBuilder to v1.3.1 and retry.

If the problem reappears, enable the debug log of PageBuilder and execute it, and post that log. I will start the PD.

• To enable the debug log of PageBuilder:
  Uncomment this line: PB_DEBUG in PageBuilder.h
  https://github.com/Hieromon/PageBuilder/blob/6a73c1bdba9fd59508f266a0a4c4127812a6a5e2/src/PageBuilder.h#L34

2. /list returns BAD ARGS

A URI of /list needs a query string. If a query string is missing, BAD ARGS will be responded.
The primitive of FSBrowser was imported from the ESP8266WebServer library. FSBrowser/list is needs a dir=[path] query.
https://github.com/esp8266/Arduino/blob/848fbf5b4a51bfd2ec58204b1262c26f5ac54af1/libraries/ESP8266WebServer/examples/FSBrowser/FSBrowser.ino#L175

However, since the original FSBrowser has been updated, I will merge into AutoConnect v0.9.7.

[GeorgeFlorian]

I've updated PageBuilder to 1.3.1

/edit still returns FileNotFound

10:26:01.651 ->
10:26:01.651 -> rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
10:26:01.651 -> configsip: 0, SPIWP:0xee
10:26:01.651 -> clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
10:26:01.685 -> mode:DIO, clock div:1
10:26:01.685 -> load:0x3fff0018,len:4
10:26:01.685 -> load:0x3fff001c,len:1100
10:26:01.685 -> load:0x40078000,len:10088
10:26:01.685 -> load:0x40080400,len:6380
10:26:01.685 -> entry 0x400806a4
10:26:02.979 ->
10:26:03.145 -> FS File: /logo.png, size: 3.80KB
10:26:03.145 -> FS File: /login.css, size: 1.62KB
10:26:03.145 -> FS File: /index.html, size: 880B
10:26:03.178 -> FS File: /back-image.jpg, size: 146.42KB
10:26:03.212 ->
10:26:03.378 -> [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 0 - WIFI_READY
10:26:03.378 -> [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 2 - STA_START
10:26:03.477 -> [AC] WiFi.config(IP=0.0.0.0, Gateway=0.0.0.0, Subnetmask=0.0.0.0, DNS1=0.0.0.0, DNS2=0.0.0.0) [AC] WiFi.begin()
10:26:03.510 -> [AC] Connecting....[D][WiFiGeneric.cpp:342] _eventCallback(): Event: 4 - STA_CONNECTED
10:26:04.440 -> [D][WiFiGeneric.cpp:342] _eventCallback(): Event: 7 - STA_GOT_IP
10:26:04.440 -> [D][WiFiGeneric.cpp:385] _eventCallback(): STA IP: 192.168.1.103, MASK: 255.255.255.0, GW: 192.168.1.1
10:26:04.706 -> [AC] established IP:192.168.1.103
10:26:04.706 -> [AC] http server started
10:26:04.706 -> HTTP server started
10:26:04.706 ->
10:26:04.706 -> Connected! IP address: 192.168.1.103
10:26:04.706 -> Open http://esp32fs.local/edit to see the file browser
10:26:22.267 -> [AC] /_ac
10:26:22.267 -> [AC] Page[/_ac] allocated
10:26:22.267 -> [AC] 192.168.1.103/_ac
10:26:22.267 -> [PB] at leaving build: 260260 free
10:26:22.267 -> [PB] Res:200, Chunked:0
10:26:22.267 -> [PB] Free heap:262948, content len.:6643
10:26:22.300 -> [PB] Transfer-Encoding:chunked
10:26:22.300 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:26:45.707 -> handleFileRead: /edit.htm
10:26:45.872 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:26:46.106 -> [AC] esp32fs.local/favicon.ico
10:26:46.106 -> [AC] Page[] allocated
10:26:46.106 -> [E][WebServer.cpp:602] _handleRequest(): request handler not found
10:26:46.106 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:26:46.106 -> [AC] esp32fs.local/_ac
10:26:46.139 -> [AC] Page[/_ac] allocated
10:26:46.139 -> [AC] 192.168.1.103/_ac
10:26:46.139 -> [PB] at leaving build: 259968 free
10:26:46.139 -> [PB] Res:200, Chunked:0
10:26:46.139 -> [PB] Free heap:262736, content len.:6643
10:26:46.139 -> [PB] Transfer-Encoding:chunked
10:26:46.172 -> [E][WiFiClient.cpp:346] write(): 104
10:27:08.145 -> handleFileRead: /edit.htm
10:27:08.311 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:27:08.410 -> [AC] esp32fs.local/favicon.ico
10:27:08.444 -> [AC] Page[] allocated
10:27:08.444 -> [E][WebServer.cpp:602] _handleRequest(): request handler not found
10:27:08.444 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:27:08.444 -> [AC] esp32fs.local/_ac
10:27:08.444 -> [AC] Page[/_ac] allocated
10:27:08.444 -> [AC] 192.168.1.103/_ac
10:27:08.477 -> [PB] at leaving build: 259664 free
10:27:08.477 -> [PB] Res:200, Chunked:0
10:27:08.477 -> [PB] Free heap:262440, content len.:6643
10:27:08.477 -> [PB] Transfer-Encoding:chunked
10:27:08.477 -> [E][WiFiClient.cpp:346] write(): 104
10:27:13.455 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:27:13.854 -> [AC] esp32fs.local/favicon.ico
10:27:13.854 -> [AC] Page[] allocated
10:27:13.854 -> [E][WebServer.cpp:602] _handleRequest(): request handler not found
10:27:13.854 -> [D][WiFiClient.cpp:463] connected(): Disconnected: RES: 0, ERR: 128
10:27:18.866 -> [AC] esp32fs.local/_ac
10:27:18.866 -> [AC] Page[/_ac] allocated
10:27:18.866 -> [AC] 192.168.1.103/_ac
10:27:18.899 -> [PB] at leaving build: 259372 free
10:27:18.899 -> [PB] Res:200, Chunked:0
10:27:18.899 -> [PB] Free heap:262132, content len.:6643
10:27:18.899 -> [PB] Transfer-Encoding:chunked
10:27:18.899 -> [E][WiFiClient.cpp:346] write(): 104

I also have no idea how to fix '/list returns BAD ARGS'. I don't know how to add a query string.

Also, please tell me if it's possible to customize the AutoConnect Menu.

Thank you !

[Hieromon]

Sorry, I checked the other source. Ignore the following comments. I'll try reproduce a issue .

The log says esp32fs.local. Is it use mDNS?
If so, please post your sketch. I have to reprocude the issue with actual sketch.

Also,

I also have no idea how to fix '/list returns BAD ARGS'

/list?dir=[path]
[path] : File path on SPIFFS.
ex. http://192.168.1.103/list?dir=/

[GeorgeFlorian]

Also,

/list?dir=[path]
[path] : File path on SPIFFS.
ex. http://192.168.1.103/list?dir=/

When trying this 'http://192.168.1.103/list?dir=/' the result is this. Is this the correct result ?

And just in case, here is the code:

/*
  FSWebServer - Example WebServer with SPIFFS backend for esp8266
  Copyright (c) 2015 Hristo Gochkov. All rights reserved.
  This file is part of the ESP8266WebServer library for Arduino environment.
*/

#include <FS.h>
#include <WiFi.h>
#include <WiFiClient.h>
#include <WebServer.h>
#include <ESPmDNS.h>
#include <SPIFFS.h>
#include <AutoConnect.h>

#define DBG_OUTPUT_PORT Serial
#define LED 2

const char* ssid = "wifi-ssid";
const char* password = "wifi-password";
const char* host = "esp32fs";

WebServer server(80);


AutoConnect portal(server);
//holds the current upload
File fsUploadFile;

bool startCP(IPAddress ip) {
    digitalWrite(LED, HIGH);
    DBG_OUTPUT_PORT.println("Captive Portal started ! IP: " + WiFi.localIP().toString());
    return true;
  }

//format bytes
String formatBytes(size_t bytes){
  if (bytes < 1024){
    return String(bytes)+"B";
  } else if(bytes < (1024 * 1024)){
    return String(bytes/1024.0)+"KB";
  } else if(bytes < (1024 * 1024 * 1024)){
    return String(bytes/1024.0/1024.0)+"MB";
  } else {
    return String(bytes/1024.0/1024.0/1024.0)+"GB";
  }
}

String getContentType(String filename){
  if(server.hasArg("download")) return "application/octet-stream";
  else if(filename.endsWith(".htm")) return "text/html";
  else if(filename.endsWith(".html")) return "text/html";
  else if(filename.endsWith(".css")) return "text/css";
  else if(filename.endsWith(".js")) return "application/javascript";
  else if(filename.endsWith(".png")) return "image/png";
  else if(filename.endsWith(".gif")) return "image/gif";
  else if(filename.endsWith(".jpg")) return "image/jpeg";
  else if(filename.endsWith(".ico")) return "image/x-icon";
  else if(filename.endsWith(".xml")) return "text/xml";
  else if(filename.endsWith(".pdf")) return "application/x-pdf";
  else if(filename.endsWith(".zip")) return "application/x-zip";
  else if(filename.endsWith(".gz")) return "application/x-gzip";
  return "text/plain";
}

bool handleFileRead(String path){
  DBG_OUTPUT_PORT.println("handleFileRead: " + path);
  if(path.endsWith("/")) path += "index.htm";
  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  if(SPIFFS.exists(pathWithGz) || SPIFFS.exists(path)){
    if(SPIFFS.exists(pathWithGz))
      path += ".gz";
    File file = SPIFFS.open(path, "r");
    size_t sent = server.streamFile(file, contentType);
    file.close();
    return true;
  }
  return false;
}

void handleFileUpload(){
  if(server.uri() != "/edit") return;
  HTTPUpload& upload = server.upload();
  if(upload.status == UPLOAD_FILE_START){
    String filename = upload.filename;
    if(!filename.startsWith("/")) filename = "/"+filename;
    DBG_OUTPUT_PORT.print("handleFileUpload Name: "); DBG_OUTPUT_PORT.println(filename);
    fsUploadFile = SPIFFS.open(filename, "w");
    filename = String();
  } else if(upload.status == UPLOAD_FILE_WRITE){
    //DBG_OUTPUT_PORT.print("handleFileUpload Data: "); DBG_OUTPUT_PORT.println(upload.currentSize);
    if(fsUploadFile)
      fsUploadFile.write(upload.buf, upload.currentSize);
  } else if(upload.status == UPLOAD_FILE_END){
    if(fsUploadFile)
      fsUploadFile.close();
    DBG_OUTPUT_PORT.print("handleFileUpload Size: "); DBG_OUTPUT_PORT.println(upload.totalSize);
  }
}

void handleFileDelete(){
  if(server.args() == 0) return server.send(500, "text/plain", "BAD ARGS");
  String path = server.arg(0);
  DBG_OUTPUT_PORT.println("handleFileDelete: " + path);
  if(path == "/")
    return server.send(500, "text/plain", "BAD PATH");
  if(!SPIFFS.exists(path))
    return server.send(404, "text/plain", "FileNotFound");
  SPIFFS.remove(path);
  server.send(200, "text/plain", "");
  path = String();
}

void handleFileCreate(){
  if(server.args() == 0)
    return server.send(500, "text/plain", "BAD ARGS");
  String path = server.arg(0);
  DBG_OUTPUT_PORT.println("handleFileCreate: " + path);
  if(path == "/")
    return server.send(500, "text/plain", "BAD PATH");
  if(SPIFFS.exists(path))
    return server.send(500, "text/plain", "FILE EXISTS");
  File file = SPIFFS.open(path, "w");
  if(file)
    file.close();
  else
    return server.send(500, "text/plain", "CREATE FAILED");
  server.send(200, "text/plain", "");
  path = String();
}

void handleFileList() {
  if (!server.hasArg("dir")) {
    server.send(500, "text/plain", "BAD ARGS");
    return;
  }
  
  String path = server.arg("dir");
  DBG_OUTPUT_PORT.println("handleFileList: " + path);
  File root = SPIFFS.open(path);

  path = String();

  String output = "[";

  if(root.isDirectory()){
    File file = root.openNextFile();
    while(file){
      if (output != "[") {
        output += ',';
      }
      output += "{\"type\":\"";
      output += (file.isDirectory()) ? "dir" : "file";
      output += "\",\"name\":\"";
      output += String(file.name()).substring(1);
      output += "\"}";
      file = root.openNextFile();
    }
  }
  
  output += "]";
  server.send(200, "text/json", output);
}

void setup(void){
  delay(1000);
  DBG_OUTPUT_PORT.begin(115200);
  DBG_OUTPUT_PORT.print("\n");
  DBG_OUTPUT_PORT.setDebugOutput(true);
  
  SPIFFS.begin();
  {
    File root = SPIFFS.open("/");
    File file = root.openNextFile();
    while(file){
      String fileName = file.name();
      size_t fileSize = file.size();
      DBG_OUTPUT_PORT.printf("FS File: %s, size: %s\n", fileName.c_str(), formatBytes(fileSize).c_str());
      file = root.openNextFile();
    }
    DBG_OUTPUT_PORT.printf("\n");
  }
  

  //WIFI INIT
  //DBG_OUTPUT_PORT.printf("Connecting to %s\n", ssid);

  //SERVER INIT
  //list directory
  server.on("/list", HTTP_GET, handleFileList);
  //load editor
  server.on("/edit", HTTP_GET, [](){
    if(!handleFileRead("/edit.htm")) server.send(404, "text/plain", "FileNotFound");
  });
  //create file
  server.on("/edit", HTTP_PUT, handleFileCreate);
  //delete file
  server.on("/edit", HTTP_DELETE, handleFileDelete);
  //first callback is called after the request has ended with all parsed arguments
  //second callback handles file uploads at that location
  server.on("/edit", HTTP_POST, [](){ server.send(200, "text/plain", ""); }, handleFileUpload);

  //called when the url is not defined here
  //use it to load content from SPIFFS
  portal.onNotFound([](){
    if(!handleFileRead(server.uri()))
      server.send(404, "text/plain", "FileNotFound");
  });

  //get heap status, analog input value and all GPIO statuses in one json call
  server.on("/all", HTTP_GET, [](){
    String json = "{";
    json += "\"heap\":"+String(ESP.getFreeHeap());
    json += ", \"analog\":"+String(analogRead(A0));

    json += ", \"gpio\":" + String((uint32_t)(0));
    
    json += "}";
    server.send(200, "text/json", json);
    json = String();
  });

  pinMode(LED,OUTPUT);
  digitalWrite(LED, LOW);
  portal.onDetect(startCP);
  
  if (portal.begin()) {
    digitalWrite(LED, LOW);
  }
  
  DBG_OUTPUT_PORT.println("HTTP server started");

  //Relocation as follows to make AutoConnect recognition.
  DBG_OUTPUT_PORT.println("");
  DBG_OUTPUT_PORT.print("Connected! IP address: ");
  DBG_OUTPUT_PORT.println(WiFi.localIP());

  //Relocation as follows to make AutoConnect recognition.
  MDNS.begin(host);
  DBG_OUTPUT_PORT.print("Open http://");
  DBG_OUTPUT_PORT.print(host);
  DBG_OUTPUT_PORT.println(".local/edit to see the file browser");
}
 
void loop(void){

  portal.handleClient();
}

[Hieromon]

Is this the correct result ?

It looks to me correctly because your sketch code says it. /list returns JSON description.
Your SPIFFS contains login.cs, index.html, logo.png, back-image.jpg.
But edit.htm is not contained. The cause of FileNotFound is this.

A handleFileList will be invoked from a uri of /list and returns contained files list in SPIFFS with JSON.

void handleFileList() {
  if (!server.hasArg("dir")) {
    server.send(500, "text/plain", "BAD ARGS");
    return;
  }
  
  String path = server.arg("dir");
  DBG_OUTPUT_PORT.println("handleFileList: " + path);
  File root = SPIFFS.open(path);

  path = String();

  String output = "[";

  if(root.isDirectory()){
    File file = root.openNextFile();
    while(file){
      if (output != "[") {
        output += ',';
      }
      output += "{\"type\":\"";
      output += (file.isDirectory()) ? "dir" : "file";
      output += "\",\"name\":\"";
      output += String(file.name()).substring(1);
      output += "\"}";
      file = root.openNextFile();
    }
  }
  
  output += "]";
  server.send(200, "text/json", output);
}

Also, an edit.htm is missing in SPIFFS.
A /edit handler is registered the below in your code:

server.on("/edit", HTTP_GET, [](){
  if(!handleFileRead("/edit.htm")) server.send(404, "text/plain", "FileNotFound");
});

and handleFileRead as:

bool handleFileRead(String path){
  DBG_OUTPUT_PORT.println("handleFileRead: " + path);
  if(path.endsWith("/")) path += "index.htm";
  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  if(SPIFFS.exists(pathWithGz) || SPIFFS.exists(path)){
    if(SPIFFS.exists(pathWithGz))
      path += ".gz";
    File file = SPIFFS.open(path, "r");
    size_t sent = server.streamFile(file, contentType);
    file.close();
    return true;
  }
  return false;
}

Your SPIFFS does not contain edit.htm, so FileNotFound is returned.
I seem your sketch works correctly, also AutoConnect.

[GeorgeFlorian]

Wait. It should include edit.htm ? In that case I really don't understand how this works. What should I write in edit.htm ? How do I delete or create a file using this ?

Why did they bother making this FSBrowser and don't explain this really important part ?
In their sketch it just says: "Open http://esp32fs.local/edit to see the file browser" but it does not work.

I am sorry to bother you, but I am really new to this whole Arduino ESP32 thing.
I know this isn't related to the AutoConnect anymore, but if you have any idea on how this damned edit should work, please help me.

Thank you !

LATER EDIT: From what I found online the /edit should work without actually having a "edit" file. It should be a menu containing all the files. You should be able to delete by right clicking on a file and saving by pressing CTRL+S.

I also found a a "newer" version here but this one works with AsyncWebServer and your AutoConnect works with WebServer.

[Hieromon]

What should I write in edit.htm ? How do I delete or create a file using this?

It's only you to parse the code in edit.htm.

Why did they bother making this FSBrowser and don't explain this really important part?

It is not an important part. It is for the explanation for WebServer facilities. And all resources necessary for FSBrowser execution are provided. What you think is important is that you want to remodel FSBrowser, it is not the responsibility of the author of FSBrowser. Depending on how you want to remodel FSBrowser, at least FSBrowser needs files in the data subfolder.
That said, I also understand your feelings that you want to remodel FSBrowser. Well, surely the explanation is missing a little.

I found online the /edit should work without actually having a "edit" file. It should be a menu containing all the files. You should be able to delete by right clicking on a file and saving by pressing CTRL+S.
I also found a "newer" version here but this one works with AsyncWebServer and your AutoConnect works with WebServer.

So?, AutoConnect is for WiFi connection made easy. If you want to get some application with browser editor, it is not for AutoConnect facility.
The reason I included FSBrowser as an example is that AutoConnect will work if you change only the WiFi connection part of the existing sketch. It is an important purpose of AutoConnect and not to demonstrate the completed application. AutoConnect is the "library".

Can you close this issue if you are not concerned with AutoConnect's problem or feature request?

[GeorgeFlorian]

As a last thing I would like to know if it should be possible to customize AutoConnect Menu (changing colors, adding menu tiles) or if it will be implemented in the future.

Thank you !

[Hieromon]

I staged the next version of AutoConnect as v0.9.7 to the branch. I'm testing now and documentation.
With v0.9.7, you can make the custom Web pages with JSON. It will be released soon. I hope.
By click the below image, you can see the some screen. It's under the develop.

[GeorgeFlorian]

That seems nice, indeed.
But you aren't answering my question. I am interested in customizing the whole AutoConnect. Like having access to the HTML and CSS behind it.

[Hieromon]

if it should be possible to customize AutoConnect Menu

So, my first answer is the v0.9.7.

But you aren't answering my question. I am interested in customizing the whole AutoConnect. Like having access to the HTML and CSS behind it.

Your story does not show concrete details at once.
AutoConnectPage.cpp has the skeleton of CSS. You will be able to change it, but if you fix perfectly, you need to know deeply about the ESP8266WebServer library. Because AutoConnect generates HTML dynamically and the entity is a proprietary class that inherits the RequestHandler class included in the ESP8266WebServer library.
If you want to understand whole the library, you first trace the source code and grip the understanding to the program structure of AutoConnect, also PageBuilder and WebServer library based ESP8266WebServer class and the RequestHandler class. I have analyzed the source code of the ESP8266WiFi library myself then designed AutoConnect but I will not explain them here.
This place for tracking the issues and to discuss improvements for the library. There is no how-to guide to remodel the entire AutoConnect.

Your story does not fall under either of an issue or specific questions for usage, or a library requirement. So close it.