ESP wifi module

[kdschlosser]

I have Interest in this as I am very evolved in the EventGhost community. the only downside is that you would have to carry your phone with you at all times. I am interested in making software for an ESP8266 WiFi module. these Wifi modules are about the size of a dime and can run on 2 coin cells. have an 80mhz RISC processor with 512K of memory. and also have an unbelievable range. firmware can be written to them using the Arduino IDE and the code is written in C++. the Wifi Module is really geared to Telnet communication but can be adapted to do more then that. but it also does support sending and receiving web page data. I would like to know if this is of interest? and if I would be able to get some assistance in this project?

[schollz]

Hi, I love EventGhost and used it many years ago! I had know idea it was still going strong.

I would love to do what you are describing. I've already prototyped a bracelet that uses the $20 Particle Photon. The Photon's are nice because they are very easy to program. I've written an example program that uses the low-bandwidth MQTT messaging with FIND and it works great.

I would however be interested in doing something with the ESP8266's, as they are cheaper. I can certainly assist with this. You mention that firmware can be written to them with Arduino IDE - does this refer to $15 SparkFun ESP8266 Thing? I'd like to know if its possible to program the chip directly instead of buying $15-20 boards.

[kdschlosser]

Yessir you just have to use an ftdi USB to 3.3 TTL serial. The only thing is during the flashing process you don't want to use a cheap wall wart. For a couple of reasons. They consume just over an amp during flashing. And cheap wall wart the voltage will drop will cause issues. As well as A/C bounce the esp's don't like it to much. You can flash them directly using the Arduino IDE and the firmware to be written in C++ and the base "software" is keyed to network comms and is made specifically for the esp. So yeah order up some they are 1.50 USD online. I would for sake of constant flashing probably get one that is mounted on a breadboard compatable PCB with pins. For learning. And playing with. But use the really small one with a couple of coin cells and encase the thing in plastic with a strap or a necklace type thing. It should be really simple to make. If you take a 3/4 inch screw in plug for an electrical box. Put the coin cells on it and the esp on top of that put it in a form and pour in some urathane. Let it set. You now have one encased with a removable battery cover.

[kdschlosser]

I do have some questions about the API though. Does your software only report to your webserver?

Because I have someone that wants to use this. But the information he has is to get the data from a website. Now I don't want to make something up for him and run a polling loop to your server. Is there a mechanism in place have it send a notification to a specific IP and port?? And if so point me in the direction of an API for this.

I would like to make one suggestion. And that would be to expand the functionality to handle the wifi location. As you have. But make it so you can specify an AP name to listen. And if not connected to that AP have it flip to getting positional data from GPS and marry that with GeoFencing. So you cover total location awareness in one app. That would be awesome. And allow for multiple fences. And send a notification when entering And leaving a fenced area. If you couple that with EG Home Automation has truly become automatic. Because to date everything requires you to pick up the phone and press a button. How Bout if I cross fence 1 turn the heat on. And when I cross fence 2 turn the outside lights on. And when leaving shut everything down and lock up the house.

:D

[schollz]

Thanks a lot for your advice on the ESPs, I am excited to give that at try 😄. As to your questions:

Does your software only report to your webserver?

Yes, it uses polling like you said. In the future I would like to use websockets or something smarter.

Is there a mechanism in place have it send a notification to a specific IP and port?

This would be a great addition! I am actually working on this - I was thinking about IFTT or Pushbullet which would give novice users a lot more control. Both can actually be used for doing Geofencing, I think. As for GPS data, I'm also working on integrating those coordinates as an option so you can actually "see" yourself on a map.

If you couple that with EG Home Automation...

As for the home-automation, there are people using HomeAssistant.io and OpenHab which are designed to turn heat/lights on/off and they work well with FIND. I think this is the coolest feature of FIND, and what I'm most interested in doing.

[hulkco]

Hi Guys, some time ago, i ask the same question, i glad to read someone else its interest in to use a ESP8266 with find, the esp8266 is a beautiful board an very cheap to!!! LoL, you can use a arduino UNO to program them, i wrote how to a: hackster Regards

[kdschlosser]

i see there is a place to change the server find reports to. is there an API for this? or is the only way at the moment to poll your web server. i just think that in order for the thing to be semi accurate that the polling interval needs to be kind of high and some smoothing needs to take place. and i don't want to make a plugin for EG and have it beat the snot outta your servers. LOL. is there is a way to get the app to report directly to EG and not have to deal with a "middle man" i much prefer this method. because it will keep all the traffic local. and if someone forgets to pay their internet bill. it won't stop working. or if something changes outside of my control it causes support issues for me. and I already have enough on my plate rewriting almost all of the EG code. but i do still like to help out the neighbor in need. and this could also prove useful to me as well. I have a setup that is kind of similar but using bluteeth, raspberry pi's and virtualhere. but the hardware is rather costly compared to the ESP modules. and this seems to be a much better solution

[schollz]

The API for FIND is here: https://doc.internalpositioning.com/api/.

the only way at the moment to poll your web server....for the thing to be semi accurate that the polling interval needs to be kind of high...

Yes, sorry this is the current state, the web server is sent fingerprints constantly. However the polling interval is limited to how fast the phone can scan. In my experience scanning faster than 2 / second can cause the phone to crash. Polling 2 / second is really not bad at all, and unless you are tracking > 50,000 people, it will not affect the server, since the payload is very small.

is there is a way to get the app to report directly to EG....it will keep all the traffic local

No, currently the app doesn't have any of the code needed to make the computations itself. In theory, you could, but it would require a lot of porting Golang code to Android (which I'm not capable of), and also a way to distribute the learning set among devices.

However, you can setup the server locally and have everything work from LAN. I would recommend this actually, since its for home automation. In this case you just run the server on a Raspberry Pi and set your app to the address:port of the Pi. It is still polling.

[schollz]

@hulkco, Thanks for the link to the tutorial!

[kdschlosser]

do you have an icon i could use??? I have made a plugin for EG and would like to add an icon for the plugin. tho at the moment EG is very limited in the icon department at the moment (has already been updated in my text release). it has to be 16x16 LOL. and if you would like to provide a description of the FIND app i will use that as well and any help information you might want to provide.

once i get everything tested and functioning properly. you are more then welcome to add a link or add EG to a list of programs that currently work with FIND.

we are trying to get together a plugin repository of sorts for EG and put together a list of supported devices, software. it's actually a massive undertaking because it has never been monitored and EG has been around since 1997. so do it now while the program is young because putting together a complete list now is going to be impossible for us.

[schollz]

Very cool! I've never used EG, but it looks neat, I'd like to give it a try.

I just made an icon for you: https://raw.githubusercontent.com/schollz/find/master/static/img/FIND_icon.png . Let me know if that doesn't work! As for description, just use the README.md description (you are free to edit/condense it as you see fit!).

[kdschlosser]

ok now i am going to make ya work. only because EG is going to overlay that icon on top of an icon indicating a plugin i think it's not going to look so hot being a white square LOL any way you could make it with a transparent background. and it doesn't have to be 16 X 16. make it 320 X 320 and i can do the resize. . This is because the version of EG i am working on allows for any sized png to be used as icons and does all the necessary conversions and re sizing internally. And by using a larger raw version it allows for larger size to be used without it looking like doodoo. LOL also if you can outline the non transparent bits in some color. This will help to make it more noticeable when layed on top of the plugin icon. because the plugin icon is a dark blue and being that your icon is black i think it will get lost in the shuffle. I have to learn how to use the PIL library to do an outline and have it calculate what is background and turn it transparent. some dayyyyy.

[kdschlosser]

Well here is something new I found.

A Python 3 interpreter for the esp8266

https://docs.micropython.org/en/latest/esp8266/esp8266/tutorial/intro.html

Cool beans. It runs slower then writing in C++ but from what I have read it's not a problem usually. But does allow for coding C++ just like desktop Python anyways.

But you can also run code right from a webpage hosted by the esp. Thats awesome.

I just picked up a breadboard compatable esp8266

What I would like to know is. What is the API for sending fingerprint data? Is there something written up for this that is correct for 2.3? I can have something written up by tonight for the esp8266.

The awesome thing is the sending code to the esp via a webpage. Super simple updates and no need to connect anything to the esp beyond the initial flash for the interperter.

[schollz]

That is cool!

Here is the API for sending fingerprint data if you haven't seen it. The API is frozen, so it did not change in 2.3.

[kdschlosser]

ok cool. it might take a couple of days to code something up. I have to learn how to use the micro python.

I am in the middle of making a custom wx widget (float slider) I am drawing it from the ground up. with the value label that tracks with the thumb. it's kinda cool i have to deal with event generation and making the thumb have an actual point to it. and that's all that's left. so maybe another day and I will be able to get to play with the esp.

[kdschlosser]

I have a question about the API.

The API explains how to learn a new fingerprint.

But i do not see anywhere in there about having a device report the mac's and signals.

the only API reference is what is below. and that is for both learn and track. and both require a location.

I understand that when you want to learn a new location or add to a location the device must be told to do so. and the name of the location is handed to the device. and when the device transmits new fingerprint data to the server this would contain that location name. and if it does not exist the server will create the location.

but if the device is in tracking mode shouldn't it just pass the mac and signal data and the server does the rest?

this is probably a typo and it doesn't need the location in there i just want to make sure or get clarification of what is needed if it is supposed to be there.

{
   "group":"some group",
   "username":"some user",
   "location":"some place",
   "time":12309123,
   "wifi-fingerprint":[
      {
         "mac":"AA:AA:AA:AA:AA:AA",
         "rssi":-45
      },
      {
         "mac":"BB:BB:BB:BB:BB:BB",
         "rssi":-55
     }
   ]
}

thanks again

K

[schollz]

but if the device is in tracking mode shouldn't it just pass the mac and signal data and the server does the rest?

Currently, the client needs to send a POST request with the data you show. However, I could easily make a server to "do the rest" so the client just has to send along mac and signal data. This server would be a lot like https://lf.internalpositioning.com where it would simply forward on the fingerprints as a /track or a /learn fingerprint depending on the user setting. That way you would only need to send the following:

{
   "group":"some group",
   "wifi-fingerprint":[
      {
         "mac":"AA:AA:AA:AA:AA:AA",
         "rssi":-45
      },
      {
         "mac":"BB:BB:BB:BB:BB:BB",
         "rssi":-55
     }
   ]
}

The group is still required because the server would be public and needs a way of distinguishing your fingerprints from someone else.

Do you think that would be useful?

[kdschlosser]

please tell me if i am wrong in this. I may not be understanding it properly. but essentially the current setup runs like this.

each device has it's own database containing all of the fingerprint data that it has been told to learn and it has to do all of the calculations to figure out where it is and then pass that information to the server. and the server is just a go between to hand off any reported data to whoever is asking.

I am basing this all off the premise that I do not know a thing about IOT or programming and how to move data files between things. I know how to pick up my phone and dial the number and i know how to open microsoft word . is basically the limits of my knowledge. LOL

if the above assumption on how the current setup works. wouldn't it be far easier to deploy and maintain a system that worked like this.

if you decide to implement security for the server. then you could have the server tell each specific device what to do. either track, learn, or to do nothing.
if not. then the user would have to have a means to directly tell the device what to do. either this would be by picking it up physically or through some kind of a webpage or a command passed to it.

however the means in order to put the device into a learning state it would have to be furnished with the group name and the location name so that information can be included in the data sent back to the server to add the data to the proper group and make the location if it doesn't exist. add the data

if the device is in tracking mode. it would have to be furnished with a devicename or username and a groupname. and the device passes the current wifi readings the devicename/username as well as the groupname back to the server.

the server does all of the number crunching based off the data that has been learned for that group and associates a location to the devicename/username.

this kind of a setup makes it easier to maintain. If you make a change to something like the way the calculations are done or if you make a change that allows further access to specific bits of data, then all the user has to do is upgrade the server. unless for some reason the way the readings get read or the format in which the data is sent back to the server changes then the software on the devices will never need to be upgraded. I personally think this would also make you life a whole lot easier as well. far less code to maintain

and then you have backing up the data as well. that's easier because there is only one version of it and it is stored in one location.

Not to mention that IOT devices by nature are very handicapped in the department of storage space, ram, and processor speed. and also are going to be flash based storage and having data written to and from it ll the time is going to lessen the life of the device buy a lot. it seems logical because of the limitations of IOT devices to move anything that can be off of it and have all the processing and storage done centrally. this will open the doors to what can be used as a device for reporting it's location. the code footprint becomes super tiny. maybe a couple hundred lines of code.

again I really do not know the exact process of how Find currently runs. but i am assuming that each device is responsible for it's own data and has to do all it's own calculations.

if set up to be a central type of system then porting the code for the reporting devices becomes a snap.

[schollz]

@kdschlosser Yes, you are wrong in how it works currently. However your idea for a central type of system is how it currently works.

This is a quick rundown on how it works:

1. A device (phone / esp8266 / laptop) makes a WiFi scan and sees MAC addresses and RSSI values.
2. The device (phone / esp8266 / laptop) sends these MAC addresses and RSSI values to a server (typically ml.internalpositioning.com). Also sent is a "group" which can different signals from different households, and a toggle for "tracking" or "learning". (If learning, then the device also sends the name of the location it is learning).

At this point, the device (phone / esp8266 / laptop) forgets everything it knew, and saves no information. The rest of the magic happens in the server.

3. The server, ml.internalpositioning.com reads the signals and adds them to a "tracking" database or a "learning" database.
4. The server does some machine learning to compute a location based on the signals (using the "learning" database).
5. Any device, anywhere, can ask the server where a device is. The server returns the computed location of the device, based on the last information that was sent to the server.

I hope that misconception is now corrected! Please let me know if you had another question or if this doesn't make sense to you. I am really happy to answer any questions :)

[kdschlosser]

OK i get it now.
but i still do not understand how the phone / esp8266 / laptop would know the location name to pass the mac and RSSI data back to the server. to do a location calculation based on the data when using a POST /Track as the API states.

this is where my assumptions on how it operated came from. because the only way i could see it knowing that information is if it housed the data to look it up.

[schollz]

Yes, this is a good question.

The phone and the laptop can easily send a location name since it can be part of the phone App or the laptop command line program and is visible to the user. The problem is with the esp8266, since it doesn't have a user interface, the user cannot toggle them.

I think there are two solutions for this:

1. Do not allow "learning" with a ESP8266. Learning must be done with a phone app or a computer.
2. Create an intermediary server, like you mention before, which toggles the switch from "tracking" to "learning" for a given device. This is what I do for the FIND-LF (https://lf.internalpositioning.com/). This way the device always sends fingerprints to a server, which you can designate via a browser whether it is for "learning" or "tracking" and then this server sends it on to FIND.

[kdschlosser]

OK... we are getting a little fumbled up here. Let me rephrase the question

My question is not with the POST /learn I understand that the device has to be provided the location name for this.

With the POST / track.
Maybe my understanding of what this is is wrong.
This is the format in which the data coming from say the ESP has to be formatted to let the server know where it is. Since the ESP has no knowledge of a location name how can one be sent in that data?

[kdschlosser]

And the ESP can be provided a web based GUI to be able to toggle the track and learn. So no worries there.

[schollz]

Ah! Okay. For POST /track the "location" in the payload is ignored if it is sent. Sorry, this is not clear, I will try to clear that up!

[kdschlosser]

so just leave it blank.
you should remove it :-D as to avoid any other confusion hate to see you asked about it again.

[schollz]

👍 Yeah, thanks for pointing that out. Let me know if you have more questions

[kdschlosser]

I should be good to go at this point. if you want to do up an access page of sorts? or maybe someone else might be wiling to. it has to be written for python most easiest solution would be a class that i can pass callbacks to so it can request specific data

I am thinking something along these lines.

a page for for actual wifi connection settings. the usual things. AP names and signal strengths and if it is encrypted or not.
a page for entering any encryption passwords (or popup)
a page for IP settings. the usual jargon. I have to check and see if i can specify a host name and suffix with micropython
a page for setting tracking/learning
now i wanted to do something nifty and show sampling real time
i don't know of anything else but i am sure something will come up.

there can't be anything in the forum of graphics. so anything done would have to be done in code. have to see what space is left and if enough possibly put the find logo or icon.

[schollz]

For the page with actual wifi connection settings / entering any encryption password - what is this for? If it is for telling the ESP to connect to a certain WiFi network, it won't be able to work because it needs to be connected to a WiFi network before it can communicate with this server.

The rest should be easy to do.

[kdschlosser]

the pages are going to be housed on the actual ESP. and the ESP will be set up as an Access Point for first time use. You would use like a cell phone to connect to the ESP and bring up the setup page for the WiFi settings and IP settings. then once set up to connect to a local network these pages can still be accessed if you need to change the ESP's Connected Wifi or any IP settings.

[schollz]

@kdschlosser Okay, I'm on the same page now.

I'm going to take a stab at this. I am going to test it on my own ESP8266 after I figure out how to use it. What kind of ESP8266 do you have and what do you use to program it? Currently I have this one but it seems pretty expensive to buy a bunch of these, but its nice cause it has lots of pins.

[kdschlosser]

Oh and here is a link to what i just purchased for development purposes

https://www.adafruit.com/product/2471

[schollz]

Awesome! Thanks for that info.

I got the Sparkfun Thing working pretty well, although I'm having trouble with the Micropython installation - the installation works but it never shows up as an access point so I'm thinking its not flashing correctly. I got the server working with Arduino IDE, though. Are you set on Micropython, or open to using the Arduino IDE instead?

[kdschlosser]

I am really not all the proficient in C++. I am more familiar with python. But either or is fine by me. I do not know if there is a way to get RSSI data from the ESP using the Arduino IDE. I would have to check into it. Also the arduino IDE with the ESP has all kinds of odd anomalies. I am not sure about micropython. But I really do like how you can ability to bring up the interpreter right from a webpage and just copy ans paste code to update the thing or make changes if using micropython. I am going to solder the header pins right now and play around with the thing.

[uncleashi]

@kdschlosser

I think you can. Try using WiFi.RSSI()
It is included into WiFi.h header.
Give it a try. I think it would work for you. :)

[kdschlosser]

My ESP is running fine. no worries there. but it is useless to connect to the thing after first flash. the WebREPL is turned off by default.

so you have to connect via serial.

then this is kind of a goofy process if you didn't build your own flash for it. then you can edit the boot.py file directly.

but you would have to get REPL into paste mode by pressing ctrl+e
then you would paste in code that is something similar to this

import machine
bootScript = """# uncomment the 2 lines below to enable accessing the REPL via a web browser
#import webrepl
#webrepl.start()
import network
sta_if = network.WLAN(network.STA_IF)
if not sta_if.isconnected():
    print('connecting to network...')
    sta_if.active(True)
    sta_if.connect('<essid>', '<password>')
    # If using a static IP
    sta_if.ifconfig(('192.168.0.4', '255.255.255.0', '192.168.0.1', '8.8.8.8'))
    while not sta_if.isconnected():
        pass
print('network config:', sta_if.ifconfig())
"""

f = open('boot.py', 'w')
f.write(bootScript)
f.close()
machine.reset()

then you would press ctrl+d to run the pasted code

this will write the "bootScript" into the boot.py file and reboot the esp. then it should connect. if you want to enable WebREPL (accessing REPL via a browser) then uncomment the 2 lines at noted in the code above

[kdschlosser]

@uncleashi

do you know if you can change the clock speed of the esp on the fly using the Arduino IDE. i think you can only set it at flash time. and it stays that way. I am not 100% sure tho.

micropython allows for dynamically changing the clock. so if you have some pretty intense code then you can bump up the speed to 160 mhz just before that code segment gets run, and after that code segment is done turn it back down to 80mhz. this would be extremely useful if you needed a way to conserve battery life

[kdschlosser]

ok so here is the basic code for sending fingerprint data if tracking..

this is just a test version of it. I do not have my AP plugged in at the moment so i will not be able to test until later today. had to move some wiring.

import network
import ubinascii
import urequests
import utime

wifi = network.WLAN(network.STA_IF)
groupName = 'Some Group'
userName = 'Some User'
server = 'internalpositioning.com'
port = 80

url = 'http://%s:%d/track' % (server, port)
track = {
    'group': groupName,
    'username': userName,
    'location': '',
    'time': None,
    'wifi-fingerprint': []
}

while True:
    track['time'] = utime.time()
    
    for ap in wifi.scan():
        ssid, mac, channel, rssi, authMode, hidden = ap
        track['wifi-fingerprint'].append(
            dict(mac=ubinascii.hexlify(mac), rssi=int(rssi))
        )
    
    urequests.post(url, track)
    track['wifi-fingerprint'] = []

super simple... this would be written to main.py in the same manner that the boot.py is written.

all you would have to do is in the while loop add any code for other things. like handling HTTP requests or changing from tracking to learning mode.

setting the username and or groupname

[kdschlosser]

well here is a sample of the output. it didn't want to run from the main.py. not sure why i will have to investigate.

i forgot that pulling time from the ESP is not going to be time since epoch. but will be time since ESP start. so we will have to put in some kind of means to get the time. if using DHCP and a time server is specified use that. otherwise nist.time.gov always works. that is if the person has internet.

another question..

does the mac address formatting have to be hex:hex:hex: and does it have to be uppercase as well?

http://internalpositioning.com:80/track {
    'time': 4568,
    'wifi-fingerprint': [
        {
            'mac': b'c427956678b6',
            'rssi': -92
        },
        {
            'mac': b'c627956678b8',
            'rssi': -89
        },
        {
            'mac': b'ecaaa077f248',
            'rssi': -69
        },
        {
            'mac': b'ecaaa077f24a',
            'rssi': -68
        },
        {
            'mac': b'9c3426101f64',
            'rssi': -92
        },
        {
            'mac': b'ae3426101f64',
            'rssi': -91
        },
        {
            'mac': b'000e8f8bbb78',
            'rssi': -65
        },
        {
            'mac': b'600292932248',
            'rssi': -89
        },
        {
            'mac': b'60029293224a',
            'rssi': -91
        }
    ],
    'group': 'Some Group',
    'username': 'Some User',
    'location': ''
}

[kdschlosser]

@schollz

you did erase the flash on the ESP yes???
python esptool.py --port /dev/ttyUSB0 erase_flash

is this the command you used??
python esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=detect 0 esp8266-2016-05-03-v1.8.bin

also is your board a NodeMCU Board?
if so you may want to try this command
python esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=detect -fm dio 0 esp8266-2016-05-03-v1.8.bin

also you need to check and see how much flash space you have. since the flash is dependent on who made the card. it could be from 512k to 16mb. and if it's 512k then you will need to flash the thing with an alternative micropython. like the one i just got has 4mb all the other ones i have purchased in the past had 512k

[schollz]

@kdschlosser

Yep, I"ve used those commands. Specifically:

esptool.py --port /dev/ttyUSB0 erase_flash
esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=detect -fm dio 0 esp8266
esptool.py v1.2.1

which both perform without errors.

After I wait 2 minutes I do see the AP. However when I try to connect via REPL I get garbage:

>  sudo picocom --baud 115200 /dev/ttyUSB0
picocom v1.7

port is        : /dev/ttyUSB0
flowcontrol    : none
baudrate is    : 115200
parity is      : none
databits are   : 8
escape is      : C-a
local echo is  : no
noinit is      : no
noreset is     : no
nolock is      : no
send_cmd is    : sz -vv
receive_cmd is : rz -vv
imap is        :
omap is        :
emap is        : crcrlf,delbs,

Terminal ready
rl▒▒r▒▒#▒n▒▒▒
             ▒
              ▒▒p▒<▒▒▒▒8▒▒ǒ▒▒p

                              ▒nn▒▒;▒nĒ▒▒

                                         b▒#l`%

and when I try to connect via WebREPL it will Disconnect the websocket immediately (though I am able to connect to the access point itself).

I've tried googling a lot, but it seems that the Sparkfun Thing is not a popular board for using with Micropython...

Edit: I got it working!

[schollz]

@kdschlosser as to your question:

does the mac address formatting have to be hex:hex:hex: and does it have to be uppercase as well?

The mac address can be anything you want - its simply a name that the server uses to distinguish APs. The name could really be anything you want but MACs are useful because they are often unique.

Your payload looks good - don't worry about the timestamp, the FIND server will fill that in as well if its not there.

[kdschlosser]

cool the server will deal with the time instead of having to code it in. that saves some effort.

I think it will be far easier to to make something up in micropython. and also be far easier to maintain. and instructions are really simple.

I should have mentioned that when you connect with micropython i went thru 3 terminal emulators due to the garbage coming in from the ESP and the emulator disconnecting before REPL showed. LOL it also helps if you press Enter a couple of times

[hulkco]

Hello guys, I am reading all the messages to get myself informed, and I want to share the ESP8266 card I am using, it has USB (CP2104) and 16MB FLASH, for only $ 5.00, this is the link:
https://www.aliexpress.com/item/WEMOS-D1-mini-Pro-16M-bytes-external-antenna-connector-ESP8266-WIFI-Internet-of-Things-development-board/32724692514.html?spm=2114.13010608.0.0.1VvIdE

I'll start testing with MicroPython
Greetings and happy holidays.

[kdschlosser]

Sorry but due to the holidays I stalled on developing anything for the ESP. I will get back to work on it today. I just wanted to give an update

[TilBlechschmidt]

I wrote a native C++ program that runs on the ESP8266 using PlatformIO and basically scans for networks and sends them to the server. That by itself works very nice but I ran into the problem, that the learned data from the phone does not correlate with that from the ESP since the reception of the ESP seems to be much weaker than that of the phone showing fewer devices in most scans.
That might be a general problem for any implementation on the ESP chips, especially since there is no good solution for learning with them.

Am I mistaken or is the gist of a client device to scan for WiFi networks, take their MAC and RSSI and send them as a JSON string to the server? It feels like I missed something (apart from the issue with the differing reception...might attempt to create an average list over an extended period of time)

UPDATE: Interestingly enough the dashboard shows RF and SVM as 100% for the room I'm currently in

[kdschlosser]

which esp are you using. they have varying antennas but also who is the manufacturer. I have found that generics don't work as well. the adafruit huzzah has a huge amount of range. i would say about 150 ft through 3 walls. it works at the end of my driveway. I was testing this out for location data in my yard as well. something like this would be easily hooked up to something like a 12 volt supply for outdoor lighting. and I could use it as a dual purpose item. it can get temp and humidity reading. i can also use it for PIR sensors. all kinds of things at the same time. I have set a bunch up to be Access Points because where i live i only had one reference.. my own AP.. so in order to dial in the accuracy these things are great.. but also I would suggest that you create a new user for the ESP or any device for that matter. because of differing antennas and wifi chips and quality of reception is not the same on every device. so you would have to do a learn routine under a new username for each device added. this will ensure that the data being learned is for a specific device. they are not really interchangeable. this is what I have done. and it works like a champ.

[kdschlosser]

another thing that will also affect the ESPis the power supply. these things are wicked voltage sensitive. when it states 3.3 it means 3.3 not 3.2 or 3.4. so if you are powering it from a wall wart (the brick you plug into the wall) if it's a cheap one you will have fast voltage drop under load. but the worst is the ac bounce.. easiest way to check is by how much it weighs. if it's light it's not a good one. but if you want some hard numbers. slap a volt meter on the 3.3+ and the gnd while the ESP is powered up and running. and look at the voltage. but also look at whether the voltage is stable.. the bounce is easy to fix with a good sized cap.. but low voltage is not so easy. the adafruit ESP's have a regulator built onto them. so you can connect 5 volts to it as a power source with no issues. and because the regulator acts as a filter no bounce.. and since you are dropping the voltage from 5 down to 3.3 it will not have a low voltage issue either

[TilBlechschmidt]

• I am using the ESP8266-12E (AI-Thinker) and it is on a dev board labeled "ESP12E DevKit v2". I picked it up at amazon a few months ago.
  
• The PSU is on the development board and gets 5V via a USB plug.
• I thought about throwing a bunch of those boards @hulkco linked back in December into some corners to increase accuracy
• I implemented the buffer I talked about earlier (averaging the received signals) and it did increase the stability of the wifi-fingerprints object but sadly it will still require some kind of training on its own. It creates a WiFi network if it isn't connected to the one configured so that might be used for a configuration Web-UI

For anybody interested: Repo with the code for the ESP
(without any real documentation on how to use it yet but just search for PlatformIO)

[kdschlosser]

See the one in that link is from AliExpress. And is most likely a generic. But also because of the external antenna connection they will usually skimp on the onboard one. If there is one. In your picture there is an onboard but the antenna on it looks nothing like the one that is on mine. I bet if you connect an antenna to it. It would function a whole lot better.

[TilBlechschmidt]

It appears that antenna from my picture works pretty well. Whilst debugging I let the scanning process run for a few hundred iterations on both ends of my bed (I switched sides from time to time and changed the "room" the results go to when doing so). I'm now capable of detecting whether the ESP is at the head-end or the foot-end of the bed which is very impressive. The closest AP I receive is through ~3 walls diagonally one floor below me @ -61dBm:

vs

It is clearly visible that the first (yellow) AP is shifted further back by ~7dBm and some others (the other yellow spike at -70dBm on the top picture) are shifted as well, even though the distance is only roughly 2 meters.

TBH I expected room-level accuracy but this seriously impressed me there. I bet that this will degrade with different environment conditions though...have to check in a few hours again.
-- Okay so even the next day with drastically changed weather conditions tracking still works pretty well

[TilBlechschmidt]

An little update for anybody interested:
I've got a first working version of the client implementation done including a nice UI for configuration 😄
Feedback is greatly appreciated, just have a look at it over @ #152

[kdschlosser]

@TheMegaTB
Oh man I am going to look at what ya have. i started doing the same thing as you are doing. but I have gotten sidetracked on another project and have become very busy with it. I am not sure if you are familiar with EventGhost or not. Its a home automation (well really an everything automation) program. I made a very basic Find plugin for it I really need to finish it up. I have been doing so much development on the core of it. The development of it stalled for almost a decade and is now moving in a rather fast pace which is a good thing. but has caused me to not work on other things. There are a couple of people that are interested in what is going on with this project. Instead of me being the middle man It would be awesome if you would be willing to drop a line to them and inform them of your findings as well as any software you have developed. I think they would greatly appreciate it. I do not know if the people are github savvy. and if not. no big deal i will have to make some time. Thanks

[TilBlechschmidt]

Looks like EventGhost has quite a few plugins listed up there. Even though I do not have any use for it since I migrated to Linux only machines (hatching my own little windows inside a VM with its own GPU 😄) it is looking interesting.

Actually, I am currently working on a project to use the so-called state of devices (e.g. your position, the cloud cover or current sun angle) to predict what you as a human would do when you enter the room by "watching" your behavior over an extended period to preempt you touching the light switch.
I am actually curious myself whether or not it works...but regardless it requires some kind of actuation mechanism which is my client. It should be totally possible for them to hijack the data the ESP sends out (since it is just JSON encoded) to the multicast group to integrate it into the EventGhost program.

I'll let them know 👍

btw if you run the program on your ESP I'd appreciate any kind of feedback. Currently the README is a big mess so I'll clean that up in the coming days and add some small features and fixes but overall it "should" works.

[edgarmove]

Hi, ESP32 has a lot more power than esp8266 and the dev board recently hits the 8USD mark, including shipping.
https://www.aliexpress.com/item/Official-DOIT-ESP32-Development-Board-WiFi-Bluetooth-Ultra-Low-Power-Consumption-Dual-Core-ESP-32-ESP/32799954012.html?spm=2114.13010608.0.0.yb2hTi

Maybe this is a better solution to include learning on the client side.

[jbrand88]

Hello everyone, I´m woriking on ESP8266 with Arduino IDE, and I would like to implement the FIND Tracker with them. As I have A LOT of esp's, it would be great to use it as a Server and as a Client.
Do you guys have tried it already? Is there any code samples for the arduino IDE?

Tks!

[TilBlechschmidt]

@jbrand88 it's is not made for the Arduino IDE but works on the ESP8266: #152
You can, however, take a look at the tracking code and port that over to the arduino IDE which shouldn't be much of a problem whatsoever since the code should mostly be identical. The link to the tracking code is at the end of the first post.

[jbrand88]

@TheMegaTB Tks for fast reply!

Let me explain my application, than u can visualize what exactly i'm looking for:

I need to Track the positioning of animals in open field without GPS.
So, what I'm trying to do is to use ESPs in collars with little photovoltaic pannels w/ battery.

As my knowledge in programming is very low, I started with Arduino IDE and youtube to learn how to program and manipulate the ESPS (thats why i'm looking for something already made on Arduino - but i can try to learn another language too).

The initial ideia was to use trilateration (using 3 ESPs as Anchors) and 1 ESP as node (this one I wanna track). But its pretty difucult to make (cuz signals arent precises), so im trying to use some thing already made (or almost made).

With that beeing said i got some noob questions:

1 - The fingerprint is obteined by measuring signals around the pleces - so actually these signals dont need any further configuration (is that correct?)

2 - Do you think FIND would be applicable to this situation (as the places the animal is gonna pass arent all the same - and its not viable to register a lot of places).

3 - Am I in the right direction? If not, can u recomend some similar project or further reading?

Tks!

[TilBlechschmidt]

First off: I am no expert in any way everything that follows are just thoughts that came to my mind ;)
So if I got it right that you've got a vast open field where you want to track the position of multiple trackers as precise as possible then I'd say that this WiFi based approach might not be the best solution. Especially since this library is more aimed at room-based tracking instead of area based tracking (it just returns a room not a position for that matter).
Regarding your questions:

1. The fingerprint of a location is basically just a list of MAC -> RSSI correlations of received WiFi networks so yeah no additional configuration needed on that part but you'd have to train every location you want to track in which would be a deal breaker for tracking a large area especially since the signals might vary a lot based on environment conditions.

2. FIND is capable of guesstimating positions, that is if you trained one location inside a room you most likely can get the same room as a result even though you didn't train every location in that room. Because of this room-based approach I wouldn't use FIND though tbh

3. Since I experimented with it a few weeks ago you might want to look into BLE beacons. They are generally used as a static object to notify people around it about smth but you could easily turn the dime around and attach such a beacon onto one of your animals. Then just put up three or more receivers which basically record the received signal strength and UUID of the beacon and send it to a server where the received signal strengths are used to trilaterate the position. You might still get some drift but it enables you to get some reference where in between the nodes your animals are without training and those beacons last for months if not years. Obviously you could also take a beacon and replace its battery with a solar panel and some form of energy storage to get "infinite" run time. The only thing required for this system to work would be a connection between the receiving nodes to a central server. Ouh and another thing: BLE beacons can have a range of up to 450 meters (but then you really want either a huge battery or a solar panel) whereas the wifi signal might drop out after less than a hundred. Would do some testing on this though before putting it all together!

EDIT: May I ask what you want to use the tracking data for? Just curious ;)
EDIT2: Just to give some reference, here is one of the long-range beacons I talked about albeit it being out-of-stock -.-

[Yekou]

Hi Kdschlosser,

I am having the same issue as you experienced when accessing the terminal with the Sparkfun Thing. How did you fix it? You wrote:

Yep, I"ve used those commands. Specifically:

esptool.py --port /dev/ttyUSB0 erase_flash
esptool.py --port /dev/ttyUSB0 --baud 460800 write_flash --flash_size=detect -fm dio 0 esp8266
esptool.py v1.2.1

which both perform without errors.

After I wait 2 minutes I do see the AP. However when I try to connect via REPL I get garbage:

sudo picocom --baud 115200 /dev/ttyUSB0
picocom v1.7

port is : /dev/ttyUSB0
flowcontrol : none
baudrate is : 115200
parity is : none
databits are : 8
escape is : C-a
local echo is : no
noinit is : no
noreset is : no
nolock is : no
send_cmd is : sz -vv
receive_cmd is : rz -vv
imap is :
omap is :
emap is : crcrlf,delbs,

Terminal ready
rl▒▒r▒▒#▒n▒▒▒
▒
▒▒p▒<▒▒▒▒8▒▒ǒ▒▒p

                          ▒nn▒▒;▒nĒ▒▒

                                     b▒#l`%

and when I try to connect via WebREPL it will Disconnect the websocket immediately (though I am able to connect to the access point itself).

I've tried googling a lot, but it seems that the Sparkfun Thing is not a popular board for using with Micropython...

Edit: I got it working!

[kdschlosser]

I ended up flashing the thing with the Arduino IDE (ESP version) I just picked some random sketch to flash onto it. once i did that then i flashed micro python onto it and it started working just fine.

I do not know why that works. But i have done this with more then one of them and it fixes the problem each and every time.

what was your solution??