Details object vs Serial Port instance

[marcoscaceres]

The SerialPortDetails dictionary currently has the following properties:

• comName
• path
• pnpId
• manufacturer
• locationID
• vendorID
• productId
• serialNumber

However, an instance of SerialPort is currently lacking those properties. It seems annoying to have to request the available ports in order to find out the details about a port itself. You would have to find the matching object, etc. etc. which is just painful.

I suggest we add the above as attributes to instances of SerialPort itself.

[dhylands]

So all of those seem to be specific to USB-to-serial adapters and not part of serial itself.

[marcoscaceres]

Right. That's what I thought. At the system level, is all that we can expect is a path?

[dhylands]

Fundamentally, yes - just a path is all thats needed to open a serial port. Everything else is additional information that may or may not be available depending on the device.

[marcoscaceres]

Ok, so it sounds to me that listPorts() can only reliably return an array of strings that are the path names.

[reconbot]

That's right

---

Francis Gulotta
wizard@roborooter.com

On Wed, Nov 6, 2013 at 4:21 PM, Marcos Caceres notifications@github.comwrote:

Ok, so it sounds to me that listPorts() can only reliably return an array
of strings that are the path names.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-27914622
.

[marcoscaceres]

ok, cool. We are basically answering #19 :)

So, I'm going to move all those properties above to the SerialPort interface, but make them "nullable" (i.e., return null when not available).

[marcoscaceres]

Ok, so what I want to do here is have the serial port instance have an info attribute, that returns an ES6 Map compatible object. So:

var serial = new SerialPort("/some/path"); 
serial.info.forEach( (value, key) => console.log(key, value)); 

What might also work is having a .getInfo() method that returns an ES6 map of the metadata. That way, you always get a fresh copy of the metadata that can be manipulated without requiring the values of the map to be read only.

var serial = new SerialPort("/some/path")
  , customInfo = serial.getInfo(); 

customInfo.forEach( (value, key) => console.log(key, value));
customInfo.delete("manufacturer"); 
customInfo.set("vendor", "Some One");
customInfo.set(controller, {someOtherObject}) 

Option 2 above could be quite useful, IMO.

[marcoscaceres]

The SerialPortInformation map would be guaranteed to at least have path.

[fbender]

Yeah, either have an immutable object / dictionary, or have a getter that returns an instance of Map. I'm not sure however if there is any sensible use case where somebody wants to change the properties … either way that should not stop us from returning a Map.

[marcoscaceres]

@fbender so, immutable objects (proposal 1) kinda suck because they are not idiomatic: it's expected that if you have a map, it will behave like a map. Making the map immutable breaks that expectation because set, delete, etc. don't work as expected.

[reconbot]

IMHO I don't think a getter function is necessary. If we make it read only
then it's harder to extend but we can trust it's contents. I wouldn't worry
about people shooting themselves in the foot, they'll find other ways if
that's the goal.

---

Francis Gulotta
wizard@roborooter.com

On Thu, Nov 7, 2013 at 11:55 AM, Florian Bender notifications@github.comwrote:

Yeah, either have an immutable object / dictionary, or have a getter that
returns an instance of Map. I'm not sure however if there is any sensible
use case where somebody wants to change the properties … either way that
should not stop us from returning a Map.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-27983476
.

[marcoscaceres]

I'm of the position that we should change the underlying primitives as little as possible (if at all) - unless we have to. It makes the code easier to maintain, and doesn't have any side effects if, for instance, the Map prototype is extended with new functionality in the future.

[fbender]

A getter makes it clear that the data tied to the port itself is immutable (by returning a copy of the internal Map), so you can never change it globally. This makes the API look more obvious though is not necessarily required …

[marcoscaceres]

Do we know of any examples of APIs that contain a getter that returns an immutable array or map object?

I was kinda basing the design on ES-402's Intl.Collator.prototype.resolvedOptions(), which returns a fresh object every time it's called.

@rwaldron, sorry to bother again - but what would be more idiomatic here? (Please see API proposal in #20 (comment) ). Having a getter that returns a frozen Map or having a method that returns a new Map every time? ... or should the properties of the "info" object just be folded into SerialPort?

[fbender]

@marcoscaceres if you return a Map via getter, the Map does not need to be frozen – that's why I'd use a getter, to actually let it return a standard Map (or Object) that is a copy of the internal Map, so it does not need to be immutable.

[marcoscaceres]

I'm worried that people will screw with it - and that will mean that the internal map and the JS map will differ (or worst, would lead to race conditions).

[fbender]

That's the idea behind returning a copy via the getter: It does not matter if the Map gets edited. You always get the "fresh" copy from the getter, and if you mess with that copy, that's up to you.

(I worried about the same, that's why I proposed the immutable object at first. Again, this is mitigated by returning a copy via the getter.)

[fbender]

You already said that, BTW:

What might also work is having a .getInfo() method that returns an ES6 map of the metadata. That way, you always get a fresh copy of the metadata that can be manipulated without requiring the values of the map to be read only.

[rwaldron]

@fbender That's the idea behind returning a copy via the getter: It does not matter if the Map gets edited. You always get the "fresh" copy from the getter, and if you mess with that copy, that's up to you.

+1

[marcoscaceres]

Spoke to @fbender on IRC, we had a bit of confusion over terminology. When I was saying getter I meant:

//object attribute. 
Object.defineProperty(x,"someGetter", {get: function(){...}}); 

As opposed to "instance.someGetter()".

@rwaldron, are you saying we should use a proper ES getter to return a new instance every time. This, of course, means that:

fooSerial.info === fooSerial.info; // false

Which might catch people out (though that is also the same for getInfo(), but at least it's a bit more predictable that you are getting a fresh copy).

[marcoscaceres]

Using getInfo() also avoid the following gotcha (maybe, didn't help jQuery;)):

//generating redundant objects on every call. 
if (serial.info.foo === someValue && serial.info.bar === someOtherValue) {}

//vs, more efficient
var info = serial.getInfo(); 
if (info.foo === someValue && info.bar === someOtherValue) {}

there might be other gotchas with looping while trying to read the values.

[rwaldron]

The terminology mistake is that you're not using the right terminology. The thing you described above is an "accessor" property.

I'm missing context for this bombardment of snippets, but I think this is pointless:

The terminology mistake is that you're not using the right terminology. The thing you described above is an "accessor" property.

I'm missing context for this bombardment of snippets, but I think this is pointless:

var serial = new SerialPort("/some/path")
  , customInfo = serial.getInfo(); 

customInfo.forEach( (value, key) => console.log(key, value));
customInfo.delete("manufacturer"); 
customInfo.set("vendor", "Some One");
customInfo.set(controller, {someOtherObject}) 

These are just properties of that serial port object, so why over complicate?

var serial = new SerialPort("/some/path"); 

// readonly data properties, don't need to be accessors to be readonly. See example at end.
serial.comName;
serial.manufacturer;
serial.serialNumber;
serial.pnpId;
serial.locationId;
serial.vendorId;
serial.productId;

// serial instances should be Iterables, ie. implement @@iterator that 
// yields [ key, value ]
for ([k, v] of serial) {
  console.log(k, v);
}


// Using this API to connect to an Arduino Leonardo from OSX:


var leo = new SerialPort("/dev/cu.usbmodem1411");

console.dir(leo);

// output:
{ 
  comName: "/dev/cu.usbmodem1411",
  manufacturer: "Arduino LLC",
  serialNumber: "",
  pnpId: "",
  locationId: "0x14100000",
  vendorId: "0x2341",
  productId: "0x0036" 
} 


for ([k, v] of leo) {
  console.log(`${k} has a value of ${v}`);
}


// output:
comName has a value of /dev/cu.usbmodem1411
manufacturer has a value of Arduino LLC
serialNumber has a value of 
pnpId has a value of 
locationId has a value of 0x14100000
vendorId has a value of 0x2341
productId has a value of 0x0036

You could take it one step further and specify that these objects also implement keys, values, entries and then you get forEach for free.

---

This is a readonly data property example:

function Readonly() {
  Object.defineProperty(this, "foo", { 
    value: "bar" 
  });
}

var ro = new Readonly();

ro.foo; // "bar"

ro.foo = "something else";

ro.foo; // "bar"

[fbender]

👍

Simple and efficient.

[marcoscaceres]

Like it, it's why I dragged ya in here @rwaldron :)

[rwaldron]

Sorry about the weird copy/paste shenanigans... I wrote that in an editor and pasted back, whoops!

[marcoscaceres]

So the problem remains that we can't guarantee the availability and consistency of the information of the serial port (except for path) - because of legacy RS232 support. Hence the original design of not having the properties on the Serial Interface and instead having the SerialPortDetails Map be populated on request - through getInfo().

However, I think we can, to a degree merge the iterator idea with the Serial interface. So you can still do:

// serial instances is Iterable
// yields [ key, value ]
for ([k, v] of serial) {
  console.log(k, v);
}

By adding an iterator SerialPortDetails to the interface definition - which will handle the magic of making Serial port objects be iteratorable over their metadata (getInfo()).

The other option is for us to pick the standard set of attribute (as @rwaldron suggests) and set them to null. However, this still leaves us with the SerialPortDetails problem: there is lots of interesting data available depending on how you are interfacing with the serial port (USB, BlueTooth, etc.).

[rwaldron]

So make them null when there is no data for the property? Objects without a predictable shape are problematic.

[rwaldron]

So how about defining a minimal subset of properties that gets exposed on the SerialPort object, and all other (optional / platform specific) data in a Map as described above? Do we even need that data? As long as we just want to connect to a device, I think we're good with as little identifying information as possible (more data may only be helpful for the user to select the correct device to connect to, not the script).

All things considered, this is solid.

• Define a minimal subset of properties—even if that means just having port and nothing else (to begin with, the spec can be added to, but not subtracted from).
• Don't bother with the Map thing from earlier, it's unnecessary complexity.

[reconbot]

👍 to @rwaldron's suggestion

[marcoscaceres]

Sounds like a plan, @rwaldron. I'll update the spec to match.

So, going back to the OP - what is the minimal set of properties that we absolutely must have?

[rwaldron]

• path (required)
• manufacturer (not required, but must be present)

{
  path: "/dev/cu.usbserial",
  manufacturer: "Prolific Technology Inc."
}

{
  path: "/dev/cu.usbserial",
  manufacturer: ""
}

Are both valid

[larsgk]

Path is what chrome.serial.getPorts have - and that is useless to us in the
selection/detection step. We use the pnpId from node-serialport for ID.
Maybe this belongs to another issue # though - things are getting a bit
mixed here ;)
On Nov 12, 2013 8:02 PM, "Rick Waldron" notifications@github.com wrote:

• path

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-28322433
.

[marcoscaceres]

@larsgk probably right, this is really the discussion for #19

[rwaldron]

Path is what chrome.serial.getPorts have - and that is useless to us in the selection/detection step.

Can you elaborate on "useless"?

We use the pnpId from node-serialport for ID.

I'm sorry, but I'm lacking context—who is "we"?

Note that "pnpId" is empty in the examples I provided here #20 (comment). That was the exact output from node-serialport.

[fbender]

… and we are not discussion node-serialport here, but a Web API where for the most part the user selects the port(s) the script accesses, and not the script itself. Thus exposing as much as possible is a non-goal, IMHO.

As Rick said, we can always expand the API, but for now we should aim for the minimum.

[larsgk]

On Tue, Nov 12, 2013 at 8:50 PM, Rick Waldron notifications@github.comwrote:

Path is what chrome.serial.getPorts have - and that is useless to us in
the selection/detection step.

Can you elaborate on "useless"?

As Marcos mentioned, this should probably have been in #19 - but just to
elaborate on "useless" ;) ->

On Linux (at least), chrome.serial.getPorts replies with anything that
matches /dev/ttyXXX it seems - resulting in a long list of potential serial
comm paths (I think I had 20 last time I checked), where only 1 or 2 are
actual valid ports.

On top of this, we need to find which one might be bound to the device we
are looking for (PnP). For the old school D-sub serial ports, where real
PnP didn't (really) exist, I can see that presenting the user with a
selection list (of tty paths / COMn ports) might make sense, but certainly
not when we do have the extra information present in the OS to help make
a smooth connection possible, keeping it simple for the non-tech user who
just want to get their new gadget connected and found by the
(web)application they want to use with it.

Besides this, at least in our (www.empirikit.com) case, where we are dong a
science lab for school kids in developing regions, we have some experiments
that require the user to connect/disconnect several times during a session
and with different devices. They will be numbered according to arrival
time (successful USB connection) - e.g. sometimes UnitA will be on
/dev/ttyACM3 and UnitB will be on /dev/ttyACM4 and sometimes the other way
around.

As I also come from a "coding since the age of 12 and h4xing all kinds of
devices", I fully understand that in many hobby cases, just being able to
select the right path (e.g. /dev/ttyACM3 or COM4) will be just fine (as it
is for most arduino developers). However, if we want this to go to
non-tech users, who didn't grow up with understanding DOS, A: and
autoexec.bat as a prerequisite to use a simple word processor - we need to
include PnP capabilities where possible and at least make it possible for
the developers building the apps for e.g. Chromebooks or FirefoxOS (or
other WebOnly(TM) devices to come, that will not give users full access to
all kinds of device settings) to provide their users with a smooth
experience.

We use the pnpId from node-serialport for ID.

I'm sorry, but I'm lacking context—who is "we"?

Note that "pnpId" is empty in the examples I provided here #20 (comment)#20 (comment).
That was the exact output from node-serialport.

Maybe not the exact node-serialport pnpId (as it seems to be a combination
of other raw values, e.g. the iManufacturer, iSerial and more) - but then
at least provide those.

br Lars

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-28326710
.

[larsgk]

On Tue, Nov 12, 2013 at 9:00 PM, Florian Bender notifications@github.comwrote:

… and we are not discussion node-serialport here, but a Web API where for
the most part the user selects the port(s) the script accesses,

We have PnP capabilities in USB. Why degrade to something, where the user
needs to understand tech when we have a possibility to do it automatically.
If the user is my mom, telling her to select the path (e.g. /dev/ttyXXX or
COMn) is a bit like telling her to access 23.62.53.99 to go to
reddit.com.. AND the port path can change anytime if she puts in the
same device the
day after after plugging in another device first.

The geek in me says: yup - path should be fine .. I can just do a "port
crawler script cron job to find what I need" ..
The Mac guy in me (don't worry - I am normally using Linux ;)) says: why
bother .. if the OS already has PnP info.

If you replace "user" with "geek", then I am in agreement. If "user" is
avg joe user who just wants the device to be found automagically - then not
so much.

and not the script itself. Thus exposing as much as possible is a
non-goal, IMHO.

As Rick said, we can always expand the API, but for now we should aim for
the minimum.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-28327599
.

[marcoscaceres]

@larsgk, that's certainly not what @fbender was intending. Basically, what we want is something like the below (it's a prototype for MIDI selection for a different spec, but same idea):

[marcoscaceres]

Note that in the above, what could be added is an "advanced" option to allow more geeky users to explicitly provide the port, or show the ports that the user agent didn't rank as important (e.g., the ones that didn't have a manufacturer or vendor and product ID, so didn't get an icon).

[larsgk]

Hi Marcos,

From the UI mockup itself, I am not sure I get 100% how the detection and
selection would be done. As far as I remember, there is not much PnP in
the basic MIDI connection (but there might be now - it's been 20 years
since I did coding for midi devices). However, for anything USB (midi, std
serial, arduino, even our FRDM-KL25Z based solution, etc.), where we do
have PnP capabilities - we should at least give the web developers the
option to use that to give a smooth experience for their users. Maybe - in
all the specs - it could be healthy to assume that it might very soon be
used on a device, where the user (geek or not) have no control at all over
anything but the browser itself (ChromeOS and more to come) and therefore
no real way of even finding out which path (/dev/ttyXXX, COMn, ...) was
assigned to the device they just connected.

Thoughts?

br
Lars

On Tue, Nov 12, 2013 at 9:38 PM, Marcos Caceres notifications@github.comwrote:

@larsgk https://github.com/larsgk, that's certainly not what @fbenderhttps://github.com/fbenderwas intending. Basically, what we want is something like the below (it's a
prototype for MIDI selection for a different spec, but same idea):

[image: screenshot 2013-11-12 20 36 51]https://f.cloud.github.com/assets/870154/1525766/379e25ea-4bda-11e3-88a7-768456fd08b1.png

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/20#issuecomment-28330912
.

[larsgk]

On Tue, Nov 12, 2013 at 9:43 PM, Marcos Caceres notifications@github.comwrote:

Note that in the above, what could be added is an "advanced" option to
allow more geeky users to explicitly provide the port, or show the ports
that the user agent didn't rank as important (e.g., the ones that didn't
have a manufacturer or vendor and product ID, so didn't get an icon).

The actual "doConnectTo" function could still be against a path. However,
then we come back to some of that fingerprinting problem, mentioned in the
beginning, why I mentioned 2 ways of connecting:

1. the "system knows nothing" connect to a specific path and hope for the
   best (user selected - for the geeky users) .. or

2. add a listener on a part of the iManufacturer/iSerial/etc. combo a bit
   like MediaQueries for orientation, etc.
   e.g. Serial.addConnectionListener("MyCompany","MyPartialSerial", doStuff)
   (I have to check the other issue # - have been a bit overloaded with work),
   where the path never gets revealed - and nobody really cares.

   —

   Reply to this email directly or view it on GitHubhttps://github.com/Details object vs Serial Port instance  #20#issuecomment-28331321
   .

[marcoscaceres]

@larsgk the PnP details for the midi device are acquired from USB. This "Proxima Direct® USB to Midi Cable Keyboard to PC Laptop Adapter" is what I've got here at home.

I think we are all on the same page here and in agreement that we need the metadata to make decisions.

I personally think that, of the three options you presented, only 1 is really viable on the web. However, we need to do some experimentation.

[JayBeavers]

@rwaldron, to be clear 'only has a path' is true only for the older RS232 ports. RS232 ports were 'obsoleted' on the PC alongside the floppy disk and the ISA bus back in August 2000 with the introduction of the Legacy-Free PC spec. Raise your hand if the computer you're using to browse Github has a nine pin RS-232 port on the back of it. Anyone? (edit: cough, sadly, my Dell T7600 workstation does...)

In all other use cases we're discussing, the device is connected via USB (directly or indirectly) and has a mandatory USB Device Descriptor record associated with it.

Let's not optimize for the vast minority case of only having a Path available. This cuts out the ability to make intelligent decisions on what device to talk to and places a technical burden on end users with insufficient information to make an informed choice.

USB Device Properties Manufacturer and Product and Serial Number are programmable in the firmware of the USB device and can be used to convey detailed information when leveraged properly -- such as 'this device has Firmata 1.1 installed on it'. This would be very helpful for your own Johnny Five project, let's not lose this ability.

The current behavior of node-serialport to not properly expose these properly on Linux or Windows can be fixed. This isn't a limitation of what can be done, it's a limitation of the state of the code as of today. I've already filed issue node-serialport #256 to get this fixed.

We're correct that this isn't the spec for node-serialport, but let's be clear that a lot of the discussion here is being driven by the principals from the node-serialport project who have experience writing a well leveraged javascript API for serial ports. I'm assuming that like me, their motivation is to have as close as possible an API experience between the browser and nodejs, so let's not drive divergence when we don't have a strong reason.

[marcoscaceres]

@JayBeavers, that's awesome. If serialport/node-serialport#256 shows that it's possible to have vendorId, productId, serialNumber, manufacturer, and product be consistent across platforms, then we are cooking :D

I will add those, along with (platform specific) path as attributes!

[rwaldron]

After reading everything that's been posted since I last responded, I feel like clicking the "Ignore" button this repo. @larsgk Yes, I too would prefer to have all the nice human readable property values

[marcoscaceres]

@rwaldron please don't lose faith - we are trying to do our best here and nothing is set in stone. We are going to have to do some iterations to come up with a good design. Will try to find you on IRC for a chat so I can get a better understanding of your concerns.

[rwaldron]

@marcoscaceres The only technical concerns I have are complexity and it appears those have been addressed in an agreeable way.

[reconbot]

I'm goin to throw a wrench into the mix here (years later) and say that some of the port info information, like BaudRate isn't really knowable until the operation of opening a port has happened. For example, you could provide an invalid buadrate and you wont know that it's invalid until after you've opened the port, set the rate and checked it again. These syscalls are sometimes blocking and better done asynchronously.

Furthermore the information retrieved while enumerating the operating system for available ports is also often only available in a partially blocking or asynchronous fashion (interrogating services, the registry, usb, etc). Browsers may be able to cache this information by path and make them available to ports but it seems like info isn't a given.

I think an async factory function to open and return ports would work around these limitations and allow us to return open port objects full of information. I don't think it's a good idea to do it in a blocking way during a serialport's object construction.

[reillyeon]

@reconbot if you look at the example code in the explainer posted in #46 I have two new methods, navigator.serial.getPorts() and navigator.serial.requestPort() which are the only way to construct a SerialPort object and both do so asynchronously.

[marcoscaceres]

Hi @reconbot 👋, would you mind formally joining the WICG?
https://www.w3.org/community/wicg/

Your input will be invaluable as this ramps up again, so want to make sure you are formally part of the group.

[reconbot]

I'm pretty sure I have haven't I?

…

On Tue, Sep 25, 2018, 10:16 PM Marcos Cáceres ***@***.***> wrote: Hi @reconbot <https://github.com/reconbot> 👋, would you mind formally joining the WICG? https://www.w3.org/community/wicg/ Your input will be invaluable as this ramps up again, so want to make sure you are formally part of the group. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#20 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABlbrGyhjl16n58xuA5yA-k6w1mauXWks5ueuOKgaJpZM4BLZJL> .

[marcoscaceres]

Oh, apologies @reconbot. Just making sure!

[reconbot]

I wasn't sure, but I'm pretty sure now 👍