Parsing binary message sent via BLE from ESP32

Hi everyone,

I am trying to make a simple app on MIT App Inventor to be able to display multiple values sent from ESP32 on my android phone.

I use BLE on ESP32 to send the values (GPS data) to the phone.
The number of variable I would like to see on phone screen is pretty much, so instead of creating bunch of characteristics on BLE, I am sending a binary message I created with a binary structure in a single BLE characteristics. It includes different data types such as uint8 and float.

However, I struggle with finding the right way to create the blocks for parsing the binary message. I have seen in other topics that using built-in Lists and RegisterForFloats procedure for repeated only float values (up to 4) is an option, but I don't know if parsing such binary structure is even possible on App Inventor. At the end I would like to display around 20 different variables on my phone screen.

Here is a sample Arduino code I have to send the data from ESP32:

#include "HardwareSerial.h"
#include <BLEServer.h>
#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLE2902.h>

// BLE UUIDs
#define SERVICE_UUID "adc78270-9394-4fc6-9018-f3bb18a1d3e8"  // UART service UUID
#define CHARACTERISTIC_UUID_RX "adc78271-9394-4fc6-9018-f3bb18a1d3e8"  // WRITE
#define CHARACTERISTIC_UUID_TX "adc78272-9394-4fc6-9018-f3bb18a1d3e8"  // READ

// BLE SETUP
BLEServer* pServer = NULL;
BLECharacteristic* pCharacteristic;
bool deviceConnected = false;
bool oldDeviceConnected = false;

uint8_t binaryMessage[100];  // Define the binary message buffer
uint8_t modeId = 1;
float currentTime = 0;
float groundSpeed = 0;
float latitude = 0;
float longitude = 0;

class ServerCallbacks : public BLEServerCallbacks {
  void onConnect(BLEServer* pServer, esp_ble_gatts_cb_param_t* param) {
    deviceConnected = true;
    pServer->updateConnParams(param->connect.remote_bda, 0, 0x20, 0x10, 2000);
  };

  void onDisconnect(BLEServer* pServer) {
    deviceConnected = false;
  }
};

class MyCallbacks : public BLECharacteristicCallbacks {
  void onWrite(BLECharacteristic* pCharacteristic) {
    String rxCommand = pCharacteristic->getValue().c_str();
    if (!rxCommand.isEmpty()) {
      if (rxCommand.startsWith("#")) {
        try {
          // Attempt to print the command (excluding the '#' character)
          Serial.print("Received cmd: ");
          Serial.println(rxCommand.substring(1));
        } catch (...) {
          // If an exception occurs during command processing
          Serial.println("Command failed. ");
        }
      }
    }
  }
};

float generateRandomFloat(float minVal, float maxVal) {
  // Initialize the random number generator with a random seed
  randomSeed(analogRead(0));  // You can use any analog pin for randomSeed()

  // Generate a random integer within the desired range
  int randInt = random(RAND_MAX);

  // Map the random integer to a float within the specified range
  float randFloat = map(randInt, 0, RAND_MAX, minVal, maxVal);

  return randFloat;
}

void BLEconfig() {

  // Set unique BLE name based on ESP32 chip ID
  String BLEname = "ESP32";

  // Create the BLE Device
  BLEDevice::init(BLEname.c_str());

  // Create the BLE Server
  BLEServer* pServer = BLEDevice::createServer();
  pServer->setCallbacks(new ServerCallbacks());

  // Create the BLE Service
  BLEService* pService = pServer->createService(SERVICE_UUID);

  // Create a BLE Characteristic
  pCharacteristic = pService->createCharacteristic(
    CHARACTERISTIC_UUID_TX,
    BLECharacteristic::PROPERTY_READ | BLECharacteristic::PROPERTY_NOTIFY);

  // Create a BLE Descriptor to Notify
  pCharacteristic->addDescriptor(new BLE2902());

  // Create a BLE Characteristic for Commands RX
  BLECharacteristic* pCharacteristic = pService->createCharacteristic(
    CHARACTERISTIC_UUID_RX,
    BLECharacteristic::PROPERTY_WRITE);

  pCharacteristic->setCallbacks(new MyCallbacks());

  // Start the service
  pService->start();

  // Start advertising
  BLEAdvertising* pAdvertising = BLEDevice::getAdvertising();
  pAdvertising->setScanResponse(false);
  pAdvertising->setMinPreferred(0x0);  // set value to 0x00 to not advertise this parameter
  BLEDevice::startAdvertising();
}

void setup() {
  // Open serial communications
  Serial.begin(921600);
  BLEconfig();
}

void LogBinary() {

  // Calculate the total size of the binary message
  size_t messageSize = 4 * sizeof(float) + sizeof(uint8_t);

  // Create a buffer to hold the binary message
  uint8_t binaryMessage[messageSize];

  // Index to iterate through the binary message
  size_t index = 0;

  // Copy fixed-size data types
  memcpy(&binaryMessage[index], &modeId, sizeof(uint8_t));
  index += sizeof(uint8_t);
  memcpy(&binaryMessage[index], &currentTime, sizeof(float));
  index += sizeof(float);
  memcpy(&binaryMessage[index], &groundSpeed, sizeof(float));
  index += sizeof(float);
  memcpy(&binaryMessage[index], &latitude, sizeof(float));
  index += sizeof(float);
  memcpy(&binaryMessage[index], &longitude, sizeof(float));
  index += sizeof(float);
}

void loop() {

  // Disconnecting
  if (!deviceConnected && oldDeviceConnected) {
    delay(500);                   // give the bluetooth stack the chance to get things ready
    pServer->startAdvertising();  // restart advertising
    Serial.println("BLE reconnected.");
    oldDeviceConnected = deviceConnected;
  }
  // connecting
  if (deviceConnected && !oldDeviceConnected) {
    oldDeviceConnected = deviceConnected;
  }

  // Generate a random float
  currentTime = generateRandomFloat(0.0, 86400.0);
  groundSpeed = generateRandomFloat(0.0, 400.0);
  latitude = generateRandomFloat(-90.0, 90.0);
  longitude = generateRandomFloat(-180.0, 180.0);

  // Generate the log
  LogBinary();

  // SEND Data Message via BLE
  pCharacteristic->setValue(const_cast<uint8_t*>(binaryMessage), sizeof(binaryMessage));
  pCharacteristic->notify();  // Send the value to the app!
}

Any help is appreciated!

Some people send temperature and humidity in separate messages with distinctive prefixes like "t:" (for temperature) and "h:" (for humidity).
(That's YAML format.)

The AI2 Charts component can recognize these and graph them. See Bluetooth Client Polling Rate - #12 by ABG

This format is easier to synchronize and easier to read.

The reason I want to send binary messages including different data types is the size of the transferred data. As I am planning to transfer the data with a very high rate (20 Hz), it is important to me to keep the message size small.

I was able to send some data from ESP32 via BLE. The message is 10 bytes and contains:
float+float+uint8+uint8

Then on the phone, I receive the bytes in a list as:
[0, 54, 241, 71, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]

It should correspond to
9-digit float,0.00,1,0

However, I don't know how to convert them to float or integers.

image

20 messages per second?

That sounds slow enough that it leaves room to use the features of BLE to make your life easier, using a different Characteristic for each

and sending eash datum in its native data type (mostly float?), and having the BLE extension register for floats on each float characteristic, with floats arriving in the BLE ReceivedFloats event.

You would have to test incoming characteristic values to know which float just arrived in that event, an if/then/elseif ladder with at least as many rungs as your different variables.

That would keep your messages small, and relieve you of having to pack and unpack your own personal message packets.

Oh, I just noticed, you are using the Clock Timer to grab incoming bytes in BLE.
That's a pre-BLE technique.

In BLE, you Register for Floats (or uint8) on each incoming characteristic and catch the incoming data streams in BLE events.

Thank you @ABG for the suggestion.

In total, I will have around 25 float and uint8 data inside the message. That is why I wanted to go with a single binary message. In that case, I think I won't be able to have characteristics for each of those 25 variables.

Also thanks for the Clock Timer catch. It is from my previous project with BluetoothClient.

My dumb and lazy approach would be to send a piece of text for each variable, one per message, in the form
L:value
where L is any of 26 letters a-z
and value is the text form of the value.

As the text messages arrive, stash the letters and their values in a dictionary keyed by the letter.

My alternative approach was indeed sending the data as string.

Now I have ~100 bytes of strings transferred without any problem. Then, I convert each string into lists by splitting. I can select any variable as I want.

I hope it won't be an overload. I will have some time to test now.

Thanks for the help @ABG.