GithubHelp home page GithubHelp logo

khoih-prog / ethernetwebserver_ssl_stm32 Goto Github PK

View Code? Open in Web Editor NEW
14.0 4.0 10.0 2.39 MB

EthernetWebServer_STM32 is a library for STM32F/L/H/G/WB/MP1 boards running WebServer using built-in Ethernet LAN8742A, Ethernet LAN8720, W5x00 or ENC28J60 shields. It now supports Ethernet TLS/SSL Client. The library supports HTTP/HTTPS GET and POST requests, provides argument parsing, handles one client at a time. It supports Arduino STM32F/L/H/G/WB/MP1 series with 32+ Kbytes of Flash, using built-in Ethernet (Nucleo-144: F429ZI, F767ZI, Discovery: STM32F746G-DISCOVERY), or ENC28J60, W5x00 Ethernet shields. Ethernet_Generic library is used as default for W5x00. Now W5x00 can use any custom hardware / software SPI

License: GNU General Public License v3.0

C++ 22.32% Python 0.71% C 76.96% Shell 0.01%
ethernetwebserver-library stm32 http-client stm32f4 stm32f7 enc28j60 lan8742a lan8720 w5x00 nucleo-f767zi

ethernetwebserver_ssl_stm32's People

Contributors

khoih-prog avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

ethernetwebserver_ssl_stm32's Issues

Problems writing to Client using EthernetWebServer_SSL_STM32 and SdFat libraries

Environment:
Arduino IDE version: 1.8.13
STM32_MCU_based_boards: 2.0.0
Board: NUCLEO-F767ZI
Libraries: STM32duino LwIP: 2.1.2, STM32Ethernet: 1.2.0, EthernetWebServer_SSL_STM32: 1.3.0, SdFat: 2.0.6
System Information: Linux Desktop-Mint-19 5.4.0-42-generic #46~18.04.1-Ubuntu SMP Fri Jul 10 07:21:24 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Context:
I am attempting to write a file found on an SD card to a connected Ethernet client. The file opens fine, and I have no problems writing to the client. For testing purposes I also write the lines read from the file to the serial port at the same time as being written to the client. What is happening though, is that a larger portion of file contents are displayed in the serial monitor, but the browser only receives about 2900 characters. The actual file, not including the header, sent separately, is about 260000 bytes long. I ran a test sending the file's content via 8000 "client.print(...)" statements and this worked fine.

I have attached code. Anything not necessary to the basic function of the code has been stripped out to give us the minimum required to debug plus additional code for debug purposes. I am using the libraries EthernetWebServer_SSL_STM32 & SdFat. My code is derived from the WebServer example. No errors are registered during operation.

The link to the SdFat library is: https://github.com/greiman/SdFat

Serial printout at startup:

Start WebServer on NUCLEO_F767ZI, using LAN8742A Ethernet & STM32Ethernet Library
EthernetWebServer_SSL_STM32 v1.3.0
You're connected to the network, IP = 192.168.2.164

Code as follows:

#include "defines.h"
#include "SdFat.h"

/****************************************************************************************************************************
  WebServer.ino - Simple Arduino web server sample for ESP8266 AT-command shield
  For STM32F/L/H/G/WB/MP1 with built-in Ethernet LAN8742A (Nucleo-144, DISCOVERY, etc) or W5x00/ENC28J60 shield/module
  
  EthernetWebServer_SSL_STM32 is a library for STM32 using the Ethernet shields to run WebServer and Client with/without SSL
  Use SSLClient Library code from https://github.com/OPEnSLab-OSU/SSLClient
  
  Built by Khoi Hoang https://github.com/khoih-prog/EthernetWebServer_SSL_STM32
  Licensed under MIT license
 *****************************************************************************************************************************/
#define REPLY_HEADER    "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\n\r\n"
    
    void Process_Client_Request()
    {
        FsFile      file;
        char        line[500];
        char        Received_Character = '\0';
        uint32_t    Character_Index    =   0;
        uint32_t    n;
        uint32_t    File_Length_Bytes;
        uint32_t    File_Length_Lines;
    
        // listen for incoming clients
        EthernetClient client = server.available();
        
        static  ArduinoOutStream cout(Serial);
    
        
        if (client)
        {
            // an http request ends with a blank line
            bool currentLineIsBlank = true;
            
            while (client.connected())
            {
                if (client.available())
                {
                    char c = client.read();
                    Serial.write(c);
                    // if you've gotten to the end of the line (received a newline
                    // character) and the line is blank, the http request has ended,
                    // so you can send a reply
                    if (c == '\n' && currentLineIsBlank)
                    {
                        Serial.println(F("Sending response:\n"));
                        
                        // send a standard http response header
                        // use \r\n instead of many println statements to speedup data send
                        client.print(REPLY_HEADER);
                        Serial.println(F(REPLY_HEADER));
                        
                        // Open the SD card
                        if (!sd.begin(SD_CONFIG)) 
                        {
                            sd.initErrorHalt(&Serial);
                        }
                        
                        // Specify the file to read
                        char fileName[] = "test.htm";
                        
                        // Open the file to READ from the SD card
                        if (!file.open(fileName, O_READ)) 
                        {
                            sd.errorHalt(&Serial, F("open failed"));
                        }
                        
                        File_Length_Bytes = file.fileSize();
                        File_Length_Lines = 0;
    
                        Serial.print(__DATE__); Serial.print(", "); Serial.println(__TIME__);
    
                        while ((n = file.fgets(line, sizeof(line))) > 0)
                        {
                            // Write the line to the client
                            client.print(line);
    
                            // Print the line
                            Serial.print(line);
    
                            File_Length_Lines++;
                        }
                        file.close();
                        break;
                    }
                    
                    if (c == '\n')
                    {
                        // you're starting a new line
                        currentLineIsBlank = true;
                    }
                    
                    else if (c != '\r')
                    {
                        // you've gotten a character on the current line
                        currentLineIsBlank = false;
                    }
                }
            }
            // give the web browser time to receive the data
            delay(10);
            
            // close the connection:
            client.stop();
            Serial.println(F("\n\nClient disconnected"));
            
            Serial.println("\n");
            Serial.print("Total bytes read: "); Serial.println(File_Length_Bytes);
            Serial.print("Total lines read: "); Serial.println(File_Length_Lines);
            Serial.println("");
        }   
    }

Abbreviated contents of the file test.htm

<html><body>
This is line number: 1<br>
This is line number: 2<br>
This is line number: 3<br>
This is line number: 4<br>
This is line number: 5<br>

...

This is line number: 7997<br>
This is line number: 7998<br>
This is line number: 7999<br>
This is line number: 8000<br>
</html></body>

Abbreviated output to the Serial monitor:

<html><body>
This is line number: 1<br>
This is line number: 2<br>
This is line number: 3<br>
This is line number: 4<br>
This is line number: 5<br>

...

This is line number: 198<br>
This is line number: 199<br>
This is line number: 200<br>
This is line number: 201<br>
This is line number: 202<br>

Writes to the serial monitor stop here.

Abbreviated output to the web client:

<html><body>
This is line number: 1<br>
This is line number: 2<br>
This is line number: 3<br>
This is line number: 4<br>
This is line number: 5<br>

...

This is line number: 99<br>
This is line number: 100<br>
This is line number: 101<br>
This is line number: 102<br>
This is line number: 103<br>
This i

Writes to the client stop here.

These are things that I have tried:

  1. Different makes & sizes of SD cards using different formats - no change
  2. Modified to file.fgets(line, sizeof(line), "\n")) & client.write(line, countOfBytesReadWithfget); - no change
  3. I looked into https://github.com/OPEnSLab-OSU/SSLClient#implementation-gotchas. They mention changes to allow for larger output buffer sizes to the Ethernet.h file. I have determined that the code is using the ethernet.h file found here: Arduino/libraries/STM32duino_LwIP/src/netif/ethernet.h. This file does not have the options to increase the buffer size. Tried going through the various header files referenced but have not been able to find a setting for send buffering.
  4. To eliminate possible timing issues I have added a delay of varying lengths to the inner while() loop. These had an adverse effect and were again removed.
  5. I have tried client.write(line, n);, client.print(line); and cout << line; methods - identical results using all three.
  6. I tried to use the standard Arduino SD library instead of the SdFat library. I also did a client.write(myFile.read()); instead of reading to the 'line' string variable, otherwise code unchanged. Same results - output to the client stops after about 104 of the 8000 line transferred.

HTTPs to api.thingspeak.com

Describe the bug

Hi, I identified something like bug during testing of WebSSL Client example.

Example as-is works fine against www.google.com, but doesn't connect to api.thingspeak.com by server name, but connects, when I connect directly to IP 52,204,34,129 in my case, connection is establlished with warning about preference to use server name.

During debugging, and validation, I found, that CN of api.thingspeak.com certificate is *.thingspeak.com, which is most probably root cause of issue, but I haven't found place, where to reconfigure it.

Just for complete picture. I'm using standard Ethernet.2.0.0 library with my own constructor to pass right SPI instance, which is part of my bigger project

Steps to Reproduce

Steps to reproduce the behavior. Including the MRE sketches

  1. Clone example WebClient_SSL
  2. Generate trust anchors for api.thingspeak.com
  3. configure defines for card
  4. set server_host to api.thingspeak.com
  5. compile. run and check serial terminal

Expected behavior

Serial will contain Connected to api.thingspeak.com

Actual behavior

Connection failed message

When I replace line

sslClient.connect(server_host, server_port)

with
sslClient.connect(IPAddress(52,204,34,129), ssl_server_port)

Connection is established and ssl client works

Information

Please ensure to specify the following:

  • Platform.io 6.1.5
  • Board Core Generic STM32F103RE ( Bluebutton STM32F103RE in STM32duino board list)
  • Contextual information trying to replace http with https for posting data to thing speak
  • Simplest possible steps to reproduce see steps above
  • Anything that might be relevant in your opinion, such as:
    • Operating system Windows 10 22H2
    • Network configuration

Context
Activity is part of my project for Home automation gateway between sensor network based on RFM69/Radiohead and Mqtt/Openhab and Thingspeak ( this is more historical from begining of project).
Currently I have about 30 nodes in network, with rate about 800+ radio messages/hour

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.