W3C WebRTC Working Group, 1 Nov 2018

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: TPAC 2018 WebRTC meeting @Monday part 1

Description

From 08:30

A

At 8:30 in the morning.

A

Yeah now what for the record? This is the recording of the web RTC working group at t back first day.

A

Meeting scheduled, we went over that a little earlier at showing it again. We have at coffee break at 10:00, so we will be talking for one and a half hours and then we have a break. That's probably the most important ain't, no.

A

Tomorrow's meeting start again at 8:30 and after lunch tomorrow we have a few slots that are additional slots, which means that if we, if we have the have to cut discussion short because we ran out of time today, we can reschedule some for the morning for tomorrow. That might be turn out to be an advantage. So by 5:30 tomorrow we're supposed to be down. We should know what we should know what the resolution is of all the acs. We have this guest.

A

We should know how to progress on all the ideas that have been presented for future work and we should be hit the ground running and when comes the hard work before the next meeting so slide. Where state of the work group.

A

That's me talking by the way, I'm still hard, Oliver Sturm I'm, saying this working group together with Bernard, who couldn't be here, as you can see, from the a meeting spirit, he's carefully listening and will tell me if I'm completely misrepresenting. What's going on, we've been doing this stuff for seven years now, the way, seven years into a two-year project, it's time to get finished so which sauntered we have a charter that says we're going to finish. We're about to see 1.0 with high priority.

A

We're gonna define the object-oriented API based on the RTC I'm, paraphrasing the Charter, we're going to describe requirements for new use cases and we're going to address them and what we think we need for those new use. Cases are some new protocols. New debt data access functions, some which are in proposals that are will be before you before the end of this meeting, and probably some that are not.

A

The environment I'm reading out of the room is that people demand that webrtc 1.0 to just work run the same program in two browsers and get the same results as you know that sir actually true for some simple cases.

A

But as of last two end of last week, we have tried to make it a little bit more true with by switching on unified plan as the peon in Chrome Canary, which immediately broke one third of a of the users, though it's not quite there yet, but we're getting closer we're getting closer.

A

We think that the markets, as in the users and the browser vendors demand that we should be given low-level access to data, which will be a recurring theme in what with what do. We do next talks it's afternoon and tomorrow morning, and we think that the market demands.

A

That either they give enough of a shouting gasps loudly when they Euston and control surfaces that allow them to stop using SDP.

B

Is this a time for debate the.

A

Header are coming, it's it. Let's do it. Let's you got presentations, Tanaka, okay, I mean one reason: I'm not hearing that so loudly, any more might be because they given up the we given up yelling at me, because nothing happens anyway.

A

So the core specs media capture and streams, webrtc 1.0 media capture streams, as a candidate recommendation, has been stable for a year. No, it hasn't it's just that we haven't published publishing a candidate recommendation for a year. We got twenty open issues of which more than half for more than three months old we have an interpreter. Mattox is very good.

A

Relatively speaking, we have promised to be at at proposed recommendation in this quarter.

A

We're about to see 1.0, we renew the candidate accommodation when we separated the identity, expect alfred 46 opening issues of which 31 are older than 3 months. That's also obvious.

A

The inter formal, the matrix shows the love of it is, and we have a confluence map that shows implementation previous community census, that this is still a work in progress in order to figure out if anything, work with webrtc 1.0, you have to test it in the browsers you're interested in, and we are promised to have the proposed recommendation one year from now.

A

Identity got separated into its own recommendation.

A

It's got 23 open issues which is 22 over and 3 months, which is absolutely not good.

A

We have not separated the test suite for the identity from the test suite from a main spec. We should be doing that so I can't say anything about the status there and again. We have promised that it will be air and a proposed recommendation a year from now community census that we seem to be a bit slow. There.

A

Resource available, we have two editors. We will. We want to take a moment to thank very much the editors who had left the editors team not for leaving, but for everything they contributed while they were there, especially than Taylor and Adam I wish. Only then is here then thank you.

A

And there are other drafts that are managed by other editors.

A

One reminder when we're thinking about what we should do first and next, is that there is no s there's only us, as in resources, come from people being motivated to do things with things they want. They care about, and organizations sponsor people to work on this based on what they care about. So it's a gift economy.

A

If you want to have something happen, give.

A

And we need to make sure we get things done both with finishing specs and with getting users down, because otherwise people do things, but they don't do this together with us, and lack of cooperation means lack of cooperation.

A

Number of all the documents danger recorded that we're working on it.

A

There's a few documents that have been very, very quiet, but we shouldn't forget them because forgotten documents come back to Micah. This is the media capture, depth documents that I haven't seen anyone complaining about for a year or two years.

A

There's all your output devices, which I think is mostly not talked about, because nobody once they're changing any more discontent, hints.

A

Couple of minor open issues, but we release the code in chrome and it's actually used by someone, not very many people, but so on and there she dscp, which was a deemed important at the time, but instead quieten down I mean what group should have a decision either killing or advancing documents.

A

These are all stuff that this depended on a core draft, so it doesn't make sense to try to advance them beyond beyond CR, but I shouldn't forget.

B

It's mediarecorder API in scope, or it's not listed there. It's.

A

Our group, it's that second built America.

A

With attention focused for this group, this meeting finish 1.0 any any bug that's worth talking about in a group should be presented. If we manage to get a presenter we're brutally deleting everything from well yeah again there that we didn't get anyone to Shepard.

A

We're looking at new api's and we want to attend to raw media so that we know what we which direction. We recommend going we're not going to finish much of at this meeting, because things don't finish in meetings, but we should get some decisions made.

A

So that's everything I want to say about status of group, and now we can take five minutes for comments on discussion on that before before we switch to the next next part of the session.

B

B

Just the commandment I really like the first black want to get all the dogs resolved. That's really the high priority for, for me and I believe a lot of the community.

C

Yeah I'm, the only one I agree with that.

A

Yes, please contribute.

A

Me, the capstan outputs.

A

The a or the capture and streams is what I like to call it, but I know the outfit. So we got three issues that we want to discuss here and Geneva. Do you want to go to if you 5:32.

C

Yes, thank you.

C

All right, so we have some old language and respect there and.

C

The in 2016 we had our solution to iframes was to add this language that for the origin, they were supposed to present a permission request to use her from. It basically said, take the origin of the top level browsing context and combine it with the iframes origin and the purpose. There was really that it's best explained with an example and I'm showing here like. If you, the top level, is New York Times, they might have source their tech support chat. Let's say they had a tech support, chat, doesn't matter, I, really.

A

We have it, we admit the interrupt I always forget. Do we have a scribe? Okay, Thomas crabby! Thank you. So.

C

They wanna outsource web RTC in iframe right now in browsers, when they request come you'll, get a permission request that says the URL of this sub iframe wants to request permission and at the time in 2016, we thought it was important that permission only be granted in this specific context. This combination, so that neither New York Times nor in the tech support dot far in this case example, would get permission if you were to visit their individual site.

C

So this has sort of been overtaken by feature policy where there's an allow attribute on an iframe and a part of the thinking and future policy is that users don't understand on frames, so it's better to have a different model where that allows you to make new submission new permission models where you grant to the top-level domain, the one that the user understands and then that domain can, if it wants to delegate permissions to our friends that he wants and the key there being, that permission is off by default.

C

So, and this is good because it means that our combined language is now redundant and can be removed. So the conclusion is that we can remove the combined language once this PR 5 or 6 I've been merged.

C

So I should mention that, with a future policy, we'll be able to implement, in browsers permission prompts that are simpler, which is shown in the second slide, where you're effectively granting permission to the top-level domain, and none of the browser's do this yet. But this would be possible for browsers to do at this point, which is closer to what we feel users would understand. Now we don't have to do that, and it is more than having a model that makes sense actually.

A

I think here chromis has been doing this this model for a couple of years, just as a summary disobeying, the specs.

C

Well, yes, but I, don't think the specs actually cover what the prompts have to cover, but what they have to contain. Eliot's.

A

One reason why the specs are and I'm clear on the point we couldn't agree right.

C

Well, but there's no rearming, we can have feature policy and this restriction mechanism that sort of can exist separate from our decision of what we want to do with the prompts, because it might make sense some browsers might want to be more techie and say: there's the actual iframe. That's asking the formulating the you axis or not at our area, but the, but it's important to mention the chrome has supported the iframe. Allow attribute for some time and Fire Fox. 64, though, has it behind the press and we hope to ship. It.

B

Series basically doing that we using the allow attribute and the prompt is only showing the top level well, the top level origin and perfect alright, with we think for users. It's most complex thing. We can provide right, so we find with its approach. That's.

C

Great yeah I need to upgrade my OSX I could try the latest Safari.

A

Sounds like everyone's shopping? Yes, yes,.

C

C

No question: yes, a slide to shows the language that gets removed and there was a I had a second slide here, because there was sort of a question here, which is that and I'm showing the green text of that god had the permission, spec that allows us to do that in case. Someone wants to go looking for that.

C

But the interesting part was that the origin and media capture was actually never used by the request permission to use algorithm. So the question then is for the sentence that or it says we removed the part that says for the origin identified by origin. Identifier, request permission, and now it won't say that anymore though the question there is that good riddance or was that an intent here to dictate something about scope of UX I.

A

Think good riddance is that I found so no matter what the question was all.

C

Right great I guess if my question was, should the spec say anything about permission UX, but it sounds like we're good.

A

C

Right so this is a question if getusermedia should be functional in security context, only getusermedia has been around for a while and I think most browsers only implement it in HTTP, Firefox I think is the lone browser that still supports HTTP that were also happy to switch to only HTTP. The remaining question is that browsers. Currently, if you go to http they'll still see see they get youtube media method, they can call it and it will fail. It will reject with not supported error. That's alright, not allowed error.

C

I'm, not sure some browsers might differ there, but anyway. So the question here. The proposal is to limit getusermedia to secure context only, and this will go a step further, which would mean that in HTTP you wouldn't even see the method on the interface, so you would get if you were to do getusermedia in navigator, immediate devices that would be false in HTTP or we could go a step further and remove the media devices object on navigator itself.

C

In that case, you know meat and doing a console log media devices in navigator would be false, so those are two choices and I mentioned a second line here, which is that if we go with the second choice, the first choice will actually type error. We should just pick one and I think the argument for getting rid of the media devices object itself. Is that none of the other methods there make any sense absent getusermedia.

C

The only question there is. You might have some reason to enumerate the numeric devices and it's not only input devices like camera and microphone, but also output, speaker devices, which is used by said sync idea, which is the next slide and on device change as well of the hang stuff for MIDI devices.

C

I didn't think of this when I wrote the slides, but it might make sense to then look at the next slide at the same time, which it's the same proposal for set sync ID on HTML media element as they kind of go. They kind of go together now because of enumerate devices.

C

If we can go back to the first slide, but that's pretty much what we should decide should we remove only the method or the entire media devices object in HTTP. Do.

A

You have any numbers on my usage.

C

Unfortunately, no I.

C

Think the question here is more of scoping and do we want to we consider these methods as a combined interface or as individual methods and there's anyone have any use cases for using either of these in HTTP.

D

But my guess is that we should get rid of the object in its entirety in the movies to you not expose what expose as little as possible initiative any right finger printing. So.

B

We're going to the only potential issue is breaking some apps, but I believe I, hope, yeah and the the gain is you're willing. Megabases do.

E

We normally just remove face visible in the it's not in navigator, or you normally throw an exception. Well,.

C

This is the functionality of the secure context, keyword in web IDL, so I think the answer is yes. I.

B

Think it's consistent with over api's over whatever guys it's my right! So, okay, yeah.

A

Welcome this adventure, I think I think secure context actually didn't exist right. We had nothing, features, that's going on and.

C

I agree as well on it I think it makes sense to remove meeting devices object, yeah.

F

I mean I think.

C

F

Point is that we've been asking people to only use HTTP, forget user media for quite some time, so this would be very consistent with direction and particularly for things like screen sharing. It is completely scary to imagine that being used with HTTP.

A

Localhost is a secure context, sometimes sometimes I'm scared by that, but it is all over.

B

Browser can actually provide in developer menu the way to to go this often on constants I.

C

Guess there's a second question here for it's applied to, which is for set sync ID on the media element. Do we want to do the same thing now that enumerate devices will be gone or more compatibility risk here? I, don't think so, because not all browsers are supported. So we.

B

Defining setting IDs group or okay yeah.

D

So you come to setting ie if you continue like the devices right well,.

C

D, where would you get the idea from right.

D

D

E

The ID is it the track. Id heard it's.

F

Really ID that comes from enumerate devices, okay,.

E

B

There's no specific values like right now: undefined empty string, empty.

C

String means the default I put on the system, which is.

A

Where this is this, the user, you shall remove a process, but that I think speaking with my chariot on it seems that the group would seems to have consensus to remove that is to check, secure and secure context as high as possible in the in the food chain. I think.

C

So and there's also a question between set sink idea, the method and sink idea. The attribute which says I could see some sites breaking because they enumerate all attributes out of some consistency. Reason but I mean if the attributes missing it would just be undefined. Anyway, we.

A

Have the usual the usual progress for deprecation, which has all this warning some counters and all that so I think normal process for remove right. Okay decision recorded secure context for everything.

A

Now over to screen capture, this is possibly the hottest spec in the suite at the moment and as someone that it's got scary bits in it. Let's say let there are 1 2, 3, 4, 5, 6, 7 8 is used on this right and it's gonna take a while. Let's get started and icky right so.

E

A lot of things haven't been defined, yet one of the issues has been open for a long time as what we do with full-screen mode and it's a bit tricky because there's different, what's fullscreen like an application, might just do spawn a new window and go fullscreen or it might have the existing window and make that whole screen, or it might do like borderless fullscreen. That's not actually fullscreen, and so there's there's cases where you might not get the expected result.

E

The concern of this issue was, if you're doing something like PowerPoint and you start full-screen mode.

E

The the full screen will be a new window, so you'll just keep sharing like the presentation notes, which wouldn't be good problem, is there's no way for us to tell like what's the intent if an application sponsor new window, it's a new window so just to just to make it define what what what we and want to do, and what we can do is the process that, if the window that you're saying enters a full-screen mode and the track is simply resized, you don't need to change any setting.

E

And you don't want to change the sharing mode because it's the window, you're sharing and you don't want to switch to and risk leaking other information, so you're still just sharing that window. But if the window, if a new window is spawned and you're sharing not that window, then then we must. We must not share the new window and if it does go into full-screen mode, the track or the existing window that you are sharing. It may become inaccessible depending on context and there's other part of the spec saying.

E

If a window is inaccessible, it might be muted.

E

So that is if you're sharing window, if you're, sharing an application and you're aggregating windows, I, don't think we'll have a problem with full-screen mode because then that's covered.

E

So that's the proposal. A concern could be that some applications won't work, but there's not much. We can do about that. There is a related issue where I want to propose that we allow the user agent to switch, which window you're sharing with the user gesture. That could be a workaround that for now yeah any thoughts.

B

This is just about clarification right, no change of behavior. You know.

E

It's it's just undefined with what to do so. This is just clarifying for.

A

The OS no knowledgeable people, it's the OS level concept of a window entering full screen in all operating systems, or is this always dependent where the concept exists or not?.

G

As far as I know it the OS and there's nothing on Mac, OS or iOS. As far as I know,.

A

There's nothing that says that window enters full screen. No, it's always a new window in newest methods to.

G

The application I.

C

Think the matters here is that there's no surprise to the end-user I think browsers if to support I, think it's with respect for browsers to support sort of smart sources that can handle this and I think what it comes down to is whether the user is as the same application or not, and that's what we don't want to violate across any surprises like with PowerPoint.

C

If you have readers notes that are visible in one mode, that would be the extreme case, I guess, but when you go fullscreen they're not visible. That might cause concern. If you didn't intend your notes to be public, that kind of stuff, it's a difficult I.

F

Would also note that in PowerPoint you're using multiple displays, so you can actually select a display that will be used to show the slides and that would solve it as well. So I think there's confusion about what goes on, but there's there's two things happening at once. I think.

E

I think, as long as the user agent is clear to the user what they are actually sharing, it's fine I think.

C

E

So should we just treat it as a resize and clarify well.

A

What I'm hearing is that we have shrub. We have trouble finding the case where a window gets resized to full screen and because there's no such OS level concepts and full screen, they say actually it'll be fine too I mean at the moment I'm presenting full screen, except that I have another screen with different stuff on it. So it seems that not surprising the user is the key thing. Yeah.

B

I agree with her: if there's no concept of full screen, if it's very difficult to define the concept of full screen in OS, how will you do that in the spec, so.

B

I'm, fine with notes, saying: hey, that's beautiful, that's best thing about full screen and then you can do track resizing, but I must not share it. If sharing window and new for screen window a spoon, this is, by definition it's a new window. So why it's saying it's a must we're saying it's a new fullscreen window, because then what about the new window? We spoon? Why is it fullscreen special there and it should already be covered yeah.

F

I mean you're just that you should only share the window. That's selected, I think that's obvious. That's a mother window! Oh.

A

So so I'm hearing that this and good reason to avoid normative text about this but informative notes about dissipation right, yes, should be there as.

C

Long as the user perceives it as the same surface, I think worker the.

A

Thing with being cast is that you usually hook into the operating system without asking application and write what the user perceives his application dependents. But.

C

I think it would be a mistake to have normative spec language that ties to vendors hands in order to do what users expect.

A

Okay, so advice: don't ask that normative. That seems to be a confusion there and now to figure out which.

C

Device, yes, so there's there's a big PR here, waiting to get merged that will define behavior in detail of the existing constraints and would get display media and, since the one reads PR so I figured I might as well post all the text here so to weed out any questions. People might have I, don't know how in detail. I should read this, but basically the that's a table that follows the same constraint about property pattern, you'll find and getusermedia that lets the properties that apply to video display surfaces. I, don't cover audio display surfaces yeah.

C

That would be a separate PR and be I. Basically outline I follow the same model that gum uses, which is, if define constraints, get pro complicated. Because there's like three dictionaries, you started out the history. There is that they used to be a single constraints dictionary. It was too complicated and we split it up to cover constraints.

C

They have a different form and when there are constraint versus when their capability versus when they're a setting and then short a setting is the simplest one. That's only the data, so, for wit, that would be like a you long. A capability would be a you long range like a min and a max for width and a constraint would be either of those because you can set.

C

You can constrain by range, so you can constrain by I this value only which would be min and Max equals the same thing exact or an ideal value or just a number. So it goes through and defines that's all of gum here where sometimes it just describes the capability, and then you figure out, if you know the min and the max, you also know what values are valid within that range and also what min and Max would throw over constrained air on, though this one says, or with as a capability max, must reflected display.

C

Surfaces with and men must reflect, the width of the smallest access aspect for surveying representations available through down scaling by the user agent and then the same for height.

C

We won't have questions at the orient if anyone has any questions just interrupt. What.

E

Yeah, what does that mean and now scaling in from because of what.

C

This is, these are mostly written as descriptions. It's a little unclear. Sometimes, with this weather, the audience is a user of the API or the implementer, but this is sort of sometimes our spec is written in between. So what it means is that, since max is the width of the source. Basically, that means you cannot upscale mm-hm and men means that capable the job of a capability is to return the full range of values that you can expect to see and and be able to use and the smallest one would be so.

C

The implication here which is covered later and slide, is that we always preserve aspect. We'd get display media and the ujin there's allowed to downscale, as far as it is capable of down scaling. So.

E

It's it's saying you must not upscale and any few down scale. You must preserve aspect ratio. Yes,.

C

If they're, no questions on that I'll keep on going and just interrupt the full frame rate. That's why it also says because of the three different data types: that's why it describes the plane number and also when it's arranged and his same thing here. Capability max must reflect the display surfaces frame rate and men must reflect the lowest frame rate available through frame decimation by the user agent.

C

Next slide, the exact acts, aspect, ratio and width in pixels, divided by height in pixels.

C

This is the same languages in gub as income double rounded the tenth decimal place and as a setting here described, eros is setting first because it easier you'll see why, as a setting represents width divided by height, this is the current. The basically describing it just as a combination of the same values. I just described as the capability min and Max must be the current setting value, rendering this property immutable from the application of viewpoint. Basically saying.

C

Since all sources available sources have the same aspect: ish minus, plus minus a pixel- then there's no there's no way the application can modify the aspect ratio, the means it's basically fixed, except if you're sharing capturing a window and the end user resizes the window. In that case, the numbers will update I'll get to that.

C

Next one is resize mode. That's a constraint! We added fairly late. That is either a value of none, meaning that the media track contains all the bits you would need to render display in full detail which, if window device, pixel ratio is greater than one which it would be on max. Retina displays means the width and height be larger than the displaced appearance from an end-user viewpoint would suggest you'll see this I think here Firefox.

C

If you do get this by media, you get the full thing where verse verses, which will actually render twice the size on a Mac.

C

Then, if you were to just put it in a unconstrained media element and play it, it wouldn't show up as twice the size and as your desktop versus Chrome does a downscale right now, so that it looks at the right size and the other value it's crop and scale, which means that the meanness track contains an aspect preserve representation of the display surfer that has been down scaled by the user agent, but not cropped because you must not crop here.

C

So this is also explained further down and as a capability, the values mounting Karpin scale must be present, which means that user agents must support down scaling next slide.

C

The next slide any questions on this. Otherwise we could go next slide. Okay, so the these are less interesting. These are the existing constraints that existed in the spec. These are, as a setting indicates. The type of display surface that's being captured at the capability in the settings. Value must be the loan value present, rendering this property immutable from the application view point again.

C

These are effectively read-only constraint if you will a bit of misnomer but the same thing for logical surface, which is the boolean whether this is a true surface or a virtual one, and then there's the cursor constraint, which is not a readable only which you can use to specify. One of the cursor capture constraint. Values indicates if and when the cursor is included and capture display surface next slide.

C

Though, since an end-user, can we resize a window, that's being captured, some of the properties you observed through settings and capabilities may change. So this language is basically saying that when that happens, the user agent must cue a task to change those things all at once and another effect of so we inherit a lot of stuff from getusermedia that made sense for cameras and microphones that don't make so much sense for capturing screen.

C

One of them is the whole over-constrained situation, which I don't believe any browser has implemented from microphone and camera either, but they make even less sense if I could put it that way for brains, I think.

C

So, basically, what this says is: if this causes an over constrained situation, then the user agent must ignore the corporate constraints for as long as they over constrain and the user agent must not mute the track, and the user agent must not fire the over constrained event and there's a note that basically says that this spec already. This allows men and exact constraints on the get display media function where they call it. They produce a type error at the moment because those are not needed.

C

Basically, the only thing you should need and get display media is blend, values or basically ideal values or the max constraint is also useful and but at the same time we do not respect reasons. It's trickier to override the track. F apply constraints method, which means that here they may produce over constrained error or succeed depending on values, but basically, unless we further modify so that my constraints will consistently throw type error as well.

C

This is basically the spec language to say, ignore men and exact constraints for apply constraints, never throw the over constrained condition and basically move on so.

A

This is is that I think this is a little bit under specified. So I commented on APR that, because it's not clear which solution you're going to actually present to the user, when one of the constraints has been has been, they ignored. Okay,.

C

In a specific case here or I'm covering it and inside later, the only thing- that's good. Okay, let's go to next slide, then so, there's a new section on downscaling.

C

So, for the purposes of the select settings algorithm, the user agent should consider all possible combinations of downscale dimensions that preserve the aspect: ratio of the original display surface to the nearest pixel and frame rates available through frame decimation as available settings dictionaries. The downscaling and the summation effects of constraints is then effectively governed by the fitness distance algorithm.

C

So this is basically saying that we already have a fitness distance algorithm to pick from all available sources, so this spec only has to define the range I mentioned earlier from full size down to smallest, available down scaled version. If these are settings dictionaries, we just define them in settings dictionaries, and then we invoke the fitness distance algorithm. So the intent behind that is for the user agent to produce output.

C

That is close to the ideal width and height and/or, ideal frame rate when these are specified, while at all times preserving the aspect ratio of the original display surface and then I have some slides later on to describe what that would look like that.

C

The remaining language adds additional or some requirements, and there was a question about what to do. Would write know this place, though the spec now says well, this PR says the user agent should downscale by window device pixel ratio by default, unless otherwise directed by the plight constraints, so that, if you don't say anything about constraints and we just do get display media, we retina the screen. You would get the chrome behavior right now, where this downscale to look.

C

So if you were to render it unconstrained, it would have the size of the your display that you're seeing but not the detail, and if you want more than that, you can use constraints, you can interrogate the spec. You can interrogate the resize mode constraint to see whether it's crop and scale. That's one way to find out. You can get the capabilities and see that capabilities are larger than what you're getting back that kind of thing. So that would allow people to that.

C

Only people who want the full enchilada to to get all the bits they need and and I so should anyway. So but then it says, user agent must not crop the captured output and user agent must not upscale the captured output or create additional frames.

C

And the note that talks about the use of the max constraint, because downscaling now is predominantly done by the ideal constraint and the constraint fitness algorithm and it's useful, it might be useful to provide a max envelope in addition to that and I'll show that on the next slide. So for those who want to copy down the URL, there's a live demo, where I tried out the basically mimic the fitness distance algorithm in JavaScript, and it allows you to click around and it mostly stays with an ideal.

C

You click anywhere here.

C

C

It's not a full canvas, but it mostly stays in there. There's some. If you make it square, there's some that's the only time one seems to go outside the ideal area. As you go, you do a large square.

C

More squares, but they get better now, there's some educated there. If width and height are very similar, you can basically just go through evil, algorithm and and figures out the sizes with the smallest fitness distance, and you can get educators whether our multiple candidates are very similar there. That's the one where you got it to be bigger and I'm, not quite sure why that happen. So.

C

But I believe I did another demo. That I didn't include here, where you can tell in these situations that that there are multiple candidates and some of the other candidates are actually better I, believe user agents. I believe this is fine, because the gum spec actually never says never puts any requirement on the user agent to pick what it should pick of the final candidates, so I believe user agents can follow the intent here and smooth out the edge cases.

C

Though, perhaps for interrupt, you know, basically, the gum spec does not dictate in a rap on down scaling either. So I think this is good.

E

C

Well, I had different versions where I offered I did down scaling by 5 pixels at a time to pixel so time, one pixel at a time, I think right. Now it does one takes a lot of time because it just makes it slower, but it covers all the values.

C

The my suggestion is, we don't say more than we already say in this PR about that.

A

That's that sounds like it's very good good to be able to show what you're, showing.

C

Any questions on that I think that's. The last slide. I have some more slide. Just for the recap slide. I won't go over. Oh there's, also a on recommendation tomorrow: I'm not mentioning advance constraints at all as a vendor. I would like to not implement them for I get display media, but it's also not a hardship.

A

Though, to me, this looks like here reasonably well defined and I believe that the spec will have fewer bugs with this pull request and without it. So my suggestion would be for the working group to say that that we should merge this pull request and work work on remaining bugs.

A

If people are a bit dazed I can understand that I.

E

Mean it's just clarifying existing stuff right like listing the constraints or what's new? Is there something new yeah.

C

Well, what is new is that respect without this PR doesn't say anything about how constraints apply, to get display, media and screen sharing right, so you could probably deduce how it should work, but it's better to write them. Yeah.

E

C

I agree and it's one more slide on day, specifically about the retina case. If there are questions on that, but basically there was some discussion on the list. If multiple ways to do this, I might ask.

C

And so basically, question was what the width height constraints numbers. What unit are they in and the answer is, is the size of the data in pixels and the cue hair is from how I looked at how canvas draw works on retina and the answer? There was basically.

C

So the proposal for screen capture and retina is basically the same thing: double it and scaled so I'm, showing the example that the text we added, what user agents should downscale by device pixel ratio by default and I'm showing the JavaScript here you could do two. If you wanted the full thing and the good thing there is that we already have the resize mode constraint, defined income and so there's no new API needed.

A

Hearing silence I'm, suggesting that we had weird uh please be our singular- is.

E

There a race here you're getting capabilities and applying its veins, while the user is resizing their window well,.

C

That's covered by the text as says: when that happens, the user agent should you attack, which means that as far as JavaScript is concerned, the valleys won't change from under JavaScript. It might be, you know, millisecond behind. If the user is continually changing, you know it will get another value and then next task you'll get a different value. Until you stop getting funds, and at that point it will be cut up. Yeah I.

F

Would like to speak up in favor of merging this, because the spec is currently in a confusing state not very long ago. There was almost nothing no text about constraints, so it's important that we get something. Is this parameter as soon as possible? Okay,.

A

I hear consensus: let's go right.

C

So this is issue different issue issue. 37. There was a request on the loan on github to limit the browser sharing to a specific whitelist or black list of domains. So URLs, let me read it: please consider adding another constraint which will be relevant for browser a commercial product may and will need to limit screen sharing to only those tabs which were open for my white list of domains or even URLs, and vice-versa it'll be very useful to support a black list of domains. Urls any never share content.

C

If a tab is negated to this address.

C

Unfortunately, this is a form of influencing user selection, which is emulation which email it says, voila right, which a malicious site may use to direct users to sharing a specific web surface under attacker control, I vote. We do not do it well.

B

What about the case, where you have a website that just won't have.

B

C

Unfortunately, that is the highest risk surface you could share, because that means an attacker could basically say that yeah, the the the vulnerability was screen. Sharing that that most people don't find obvious is that when you're sharing the attacker web surface and the attacker can itself see its own screen, they can then launch basically any URL in an iframe and thus circumvent the cross origin protection, which means they could launch a bank URL where you're logged into, and then they could see the bits of data that they shouldn't be able to see.

C

And I have more slide so basically I copied from somewhere else. That goes into detail of this. If there were question.

B

Basically, your your guiding towards not not allowing tap share, maybe mainly I, guess well,.

C

I think what matters is that I don't think respect this allows time sharing, but the spec says that the JavaScript, the web, the drive-by web, cannot influence green sharing beyond offering the user to screen share and then which will bring up a selector that is under browser control. That can then fully explain to the user what the risks of the different choices are, and it was has always been considered harmful for to have a JavaScript surface that can narrow that choice beyond that.

B

But it seems good for us I believe really do something.

A

So close with clothes, this bag with no we're not doing that and here's why I never to read? Okay.

A

I'd, like aggregate windows right.

E

The spec says that you can. The multiple knowledge can be aggregated into a single, logical, monitor and multiple windows can be aggregated into a single application surface and again. This is just clarification because we never say what that means and I think we should say that you know the relative window. Sizes must be maintained.

E

So if you're sharing a small window and large window, you shouldn't get any weird effect and of course this is needed, but with the area between windows you're sharing this window in this window, you must not make other application data when you're, sharing the total surface. That and I think it's an open question what to do. When you have multiple windows and and there's any space in between or or same with, the resize, as shown in the demo earlier I suggest, we say it should be building black.

E

That's right, I, don't know if we want to say that the user agent has to maintain the decision of the windows like if the window. If the user agent wants to share windows side by side, I, think that's, ok, I.

A

Think the user agent should be allowed to hidden too thin and purple filling.

E

A

Black is such an ugly color.

B

The with with implementing that who is using that I would just question whether is it like something people actually won't. You have a browser, support that aggregated.

B

I'm questioning the use case so maybe.

F

B

It and then we don't have to handle.

E

Yeah I'm not sure, there's a distinction in respect between window and application in a lot.

H

E

But I think the distinction is actually confusing.

F

Application, what we edge does not support an application or aggregation.

E

I'm guessing the use case is, let's say: you're, sharing a photo shop window and there's a lot of different windows that pop up that show, but I mean if you want to hear that, why aren't you just sharing the screen?

E

It's actually even more simple. Well,.

B

I would think that if you really want basically in each window should be a track and then you have a list of tracks an array of tracks and you send them and you don't have to do black. You don't have to do anything but I'm not sure it's useful to do that. So maybe you can keep things simple, I'm not supported. Well,.

C

Firefox does support application, share and I think it's useful in the.

C

Programs that have multiple windows and I think it would be good to keep that in during the full screen, while it would have the same effect has the full screen ring. Has the secure heightened security risk because it could usually contains one so I think it's good.

D

C

Only thing probable difference here would be that there's a logical, there's, a there's, a constraint that would return the window would return it. Sometimes that says that Firefox we're shooting to downplay the that's a nice. If we're gonna have a think. Those are all applications that you can I think what we're thinking is that we're gonna go maybe in the future.

C

Wood has a lot. Maybe we can offer that in a combined sort of special surface for the user to pick and then it would be a challenge for the prompt and the selector to communicate that to the user. Yeah.

B

My fear is that, from a user perspective, it might be complex. Do you understand what the browser is doing and what the browser is sharing the screen is or a window is an aggregation where there will be things that are black, it's less simple and I'm, not sure, as you say, if we downplay it, maybe it's not happy this.

C

Back right, my concern is that for some of these I guess, maybe it's not that popular anymore to have windows with tons of toolbars but I think end-users often think of Photoshop, for instance, as a single window, including all those little things I, think that's. The reason to have it in here would just be to cover that case. I mean the alternative would be to call that a window as well. But then there are questions with applications like you know. What should you fill in?

C

What color should be pin and should be black or purple, and that kind of thing so I don't think that the constraint does any harm either yeah.

B

But if, if a browser can select proto one black and whites display in things like the browser could say, oh it's an application that I will share it also like window and nobody will know and working. It's fine.

C

Well, I, don't think this back here demands that any browser has to implement application, sharing that more of a question of whether we should have the constraint there. This is a readable, only constraint, so we could leave it in there, in which case Firefox, maybe someday. You would get the value application back or if we don't have it, and probably you will get the word owner.

E

I mention this widest the application needs. You know what type of surface it is right from the application point of view, it's a part of a screen and like that's all that any track ever is. We could also remove the constraint.

C

But it's been there for a long time.

E

I mean and oh I think I think there's another room where you know the browser the implementation can do whatever it wants from the specs point of view, stealing pixels.

E

If someone wants to implement the only thing they implement this application, it's always abrogated or another implementation wants to do only window, and it's always windows or even an implementation wants to do both with the selection. It's I don't see how it matters from application point of view. This seems more, like you serves making choices, point of view, yeah.

F

The only thing I've heard from developers, which this isn't was folklore is they wanted to know what content hint they should use, but I don't think this really gives any information it shouldn't matter, whether you do a display or an application or a window. That's not going to really help. You determine the content in so as far as I can tell it's fairly useless.

C

Well, it's been there in the spec for a long time. I think. Maybe we should ask on the list before we remove it outright. The other value said how.

F

Anybody implemented, which we chose not to I.

C

Think if you look at display capture, surface type, there's monitor window application and browser I guess, there's some value to an application to know, because we don't I, think the I think the logic goes as far as we're not allowing the JavaScript application any choice over constraining what the user ends up selecting so it seems at least we could do is tell them what kind of thing the user selected the.

E

White is thematic because.

C

It might if you're sharing a window versus an application versus full screen it might, you might have different choices of I, don't know the size.

C

Sure but there are security risks, for instance, in the case of browser, maybe that's the reason for removing the constraint.

F

Yeah, like I, said the only reasons I heard of were either inappropriate reasons or things that probably wouldn't work, such as the ability, the desire to figure out what continent to use.

A

The way I'm hearing that so this is the simplest version, is removing the constraints entirely, and second option is to remove the application value because it doesn't seem to be clearly defined enough for the be useful.

C

And the third option will be delivered. Yes,.

A

Three choices: no no constraint no application leave at it give us is.

A

Anyone want to argue for taking one of these choice of the table.

A

A

Anyone wants to argue that we should that we we should push this to the list and and not try to make not try to make a call now. I'm.

B

Fine with going to the list I would prefer in that case that we say hey working, wrap. The working group is planning to do that. If you do not hear any feedback, we will do it.

A

That's an argument for for trying to make trying to take a position now I.

E

Think the user-agent should be allowed to. You know let the user share whatever. If they want to aggregate it's fine, if they don't that's fine and but as what we show to the JavaScript I think it's just irrelevant, just like we don't say which window they're hearing it's just the track. You shouldn't say this is the type so.

A

Just the quick straw poll to check check if we can remove any of the options immediately and the three choices are remove the constraint. Remove application. Give us is remove the constraint, raise your hand.

F

I'll also speak up for removing the constraint.

A

Remove application for both things or yeah. That's a remover application, I.

F

Will also speak up for a moving application when.

E

You say when you say: remove application, are you saying just remove any language, but let the user agent do it every once or.

A

Are you saying that you're not allowed.

E

To educate, oh yes,.

A

I'm saying that the remove the value application and remove any special special mention or application is suppressed, windows. Okay, um third, one keep-keep abyss.

C

I'm raising my hand only because I want to check with Martin Thompson first, since he added the original language. Okay,.

A

I think that was actually interested in I'm trying to alleviate respect yeah, so the so at the moment, I think we got wait. We're going to take this this to list, but the two trailers wait to read with comes promise. Support seems to be application. They move advocate application or removal constraint now between remove application and remove constraint, which would you prefer I.

D

Guess I don't know what for remove the constraint, because I have a.

D

Document for we but I might.

A

So the the other values of the constraint are.

C

The other values are monitor, window application and browser.

A

So removing the constraint would remove them the ability together, we all skip to tell the difference. So this your hand for the alternatives now I remove application and remove and remove the whole constraint and you're only allowed to take your pain, wanna, remove the application.

A

Remove the constraint I'll.

F

Vote in favor of removing the constraint okay.

A

We have, we will take to the list the option of the suggestion that we should remove the entire country. Thank you.

E

Okay, this is a bit of a recap from the virtual interim with yeah, but the so when it's about the audio constraint, if you're sharing video, like you're doing a presentation, you might also wanna show like a video clip that canonical use case where you might want idea, but alright, so initially I proposed like let's make it very well-defined. You just make it simple: let's always always give all the audio, but it's it's complicated, because what audio sources are available? Is this platform dependent an implementation dependent? Sometimes you might be able to share all audio.

E

Sometimes you only have access to tabs audio. Sometimes you have a window, it's very unclear and then it's complicated to mix different audio sources, and so the conclusion was to say as little as possible about audio and another thing to note: is that audio? Is it's something complementing the video sharing? It's not you know it's not.

E

Essentially, it's not like a microphone, and so the the the decision from the interim was to to allow the audio constraint together with the video constraint, but it's up to the user agent, which audio sources, if any that entails, which means audio, is ignore. Belitz, it's an optional thing to include.

B

The thing would be that, if you is not there, they will never be any audio track and it would. You is true.

E

So, if yes, sir, if if the user agent knows that there will not be any audio for the duration of the stream, then you will not return an audio track in the in the resulting stream. But you could you could return an audio track, it's no muted for some time and then it's unmuted or you know, that's that's it's up to the user agent, but this is this. Just says that the if audio is available, there is an audio constraint and you may get an audio track, but don't count on it.

C

I'm a little confused about this line here at the top line, it says audio, true video, true zoom, on sharing audio and video. That makes sense, and then it says proposal recap: it's a.

E

Audio true only that there was just lazy writing. Okay. So so we did conclude that if you ask for audio but not video, you must reject the request. So it's always an compliments to video okay.

C

So, just to be clear, then, because get display media currently says that if you pass no arguments, you get video by default yeah. So we're saying that if you want video now that you have to specify both video, true and audio true, yes, if.

E

You want audio, you have to specify it, but you might not get it right. I.

C

Think this is important, because what you're, really the whole spec is written around, sharing, display surfaces and I, don't think we want to change that focus and getting the audio out of a display surface is a nice bonus, but nothing were.

D

F

D

Eat works. What would you hear what.

C

Do you mean so.

D

If you say get.

C

Then you get video the same as video :. True.

C

It's saying that the default is to.

D

Get particularly on machinery.

C

Well, I mean the dictionary is used here to carry information about options and I. Think it's ok to say that this method by default, the default for this, the default value, is video. True,.

E

And that's a separate discussion, but the annum. That's a good plan. Well,.

C

We couldn't remove the implicit value, I'd be okay with that.

E

Anyway, that this is sort of recap, does anyone want to object, because the Pierre has not landed I.

B

Would not object at the same time. It's it's a new feature and I'm sort of like on the edge of so I guess Chrome will implement. Ito is implementing it right.

B

Yeah should be simple enough: I guess. Hopefully it's not adding any complexity and.

I

Why don't I have a clarification with? Is the audio ice.

I

He names and things like that.

F

Yeah, my only concern would be audio was leaked from the system when the users.

J

E

There is responsibility by the user agent to make it clear to the user. What they're, sharing I think it's. It could be a valid use case to share more audio than just the particular window, but I can also see how you might not want to do that.

B

So then, you would recommend to share with you only you sharing the screen, for instance, so.

E

I think I think I think it's up to the like. As long as the user understand what they're sharing right like what what is like I, don't think any user has like a normal user would have a like. Oh what's a different audio sources, they would just think sharing audio would they would think of their own speakers like yeah I'm sharing my audio, they wouldn't think like Oh, which application is producing this audio and well.

B

It's an API to get the screen, which is clear from even now we are saying: oh, it would be audio.

C

If you share the full screen, I could see people expecting maybe audio from your system or maybe they've. They might the promise they might not expect any audio, but if they, if the user agent tells them I, you can also include the audio great if I share the full screen that might be okay with having all system audio. If I only share like PowerPoint I might not expect that other audio participant would be shared. This could.

F

Actually be a way of surveilling a PC and getting all audio out of it, even though the person just selected something some small window or something you end up, that's used for telephony end up listening in on all the phone calls to.

E

Be clear that the user agent has to to tell the user what they're sharing right like like the the whole point of making the this this audio thing optional? Is that that you know just like the application doesn't force the user which we know they should share? They shouldn't force the user which audio to share. So it's it's a choice. It's a user choice.

B

Some people, some feedback, is saying that basically adding this feature will probably need guidelines or, if fluent in terms of security and so on, I, don't know whether that had been made on that so I'm not pushing against. Maybe it will be done later on, but it will delay the whole spec as well, because this work will be needed if we include that feature in this back right.

A

This is, of course, only the media capture and screen spec of the caption mains button, so.

C

The spec does have a security section, I think at minimum. It should mention something about not over sharing audio okay and what the security risks might be with that, like correlating the like fingerprinting the user based on other tabs, they might have open that kind of stuff.

E

Yeah that definitely should have some fiction.

A

About that I know, they're closing in on the coffee break. Yeah, though I'm also doing that we have a decision here to allow the PRS written and note that we have to beef up the security section about the key about saying, sharing, risk of or sharing audio.

C

A

C

Just want to follow up I'm, assuming that we would want to have the audio constraints that we're used to having then echo cancellation, noise suppression and auto gain control, or no. No. Let's.

A

Make it simple right.

E

The coffee break our next slide.

E

Next I will be a headache. It's a follow up on this slide.

A

A

Let's make a coffee break until 10:00. 10:30 need coffee for this.

A

It ca 10:30.