W3C WebRTC Working Group, 17 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: WebRTC WG meeting 2022-05-17

Description

See also the minutes of the meeting https://www.w3.org/2022/05/17-webrtc-minutes.html

01:05 Future meetings
04:00 WebRTC
04:00 RTP Header Extension Encryption
12:00 Issue #2735: webrtc-pc does not specify what level of support is required for RFC 7728 (RTP pause)
16:00 CaptureController
36:14 WebRTC Encoded Transform
39:29 Extensions to Media Capture and Streams
39:30 PR #61: Add support for background blur and configuration change event
52:44 PR #59: Add powerEfficientPixelFormat constraint
1:02:14 Dynamic Source for Screensharing
1:35:30 WebRTC
1:35:38 Simulcast issues
1:45:36 Next meeting

A

Don't do this every day, so there.

B

A

B

Okay, this is the may meeting of the webrtc working group a little bit about the ipr policy.

B

Group of bytes by it, and only people in combination listed on this page are able to make substantive contributions.

B

So today we've got a lot of things we may not get through all of them, but uh webrtc extensions, webcpc capture controller, go to transform capture, extensions and dynamic sources.

B

We have a proposal to move the june 28th meeting to june 7.

B

next meeting after that is july, 19th and potentially cancel the august meeting, because usually people are on vacation in august. How do people feel about june 7th? Is that something we could do.

C

A little bit then.

C

Do we want to take a poll or something like that.

B

Yeah, why don't we try.

D

B

Just uh can't you know attend yes or no can you can you make it? Let's say that okay.

D

A

uh Okay, I launched it as an actual poll just to make it uh easier: okay, okay or although it looks like everybody's ears, except for single. No, so.

A

um Should we conclude this, it looks like it's uh four, yes, one, no and one. I cannot make either way so.

B

D

So yeah I'll say.

C

All right so move details.

B

Okay, all right a little bit about this meeting, uh the slides link to the slide is up on a wiki. We do need a volunteer for note-taking.

B

Do we have anybody to take notes.

E

Since we're already 15 minutes late I'll, take this up.

B

Okay, um all right, so we do have a code of conduct in w3c, uh so let's keep it professional and cordial.

B

um You know do uh for the uh speaking uh use, plus q and minus q in the google me chat to get into and out of the speaker queue. So we can manage that speakers can manage their own cues when you're doing your slides. uh I won't go over this understanding documents stuff.

B

Okay, so um I think what I'll do uh dom to give more time, I'll kind of only probably only deal with like one or two issues from the webrtc extension stuff, and then we can move to capture controller, to kind of get ourselves back on track.

B

Okay. So uh this is the list of stuff that I have on the list. We probably won't get too much more than two of them, but let's start with rtp header extension encryption, so uh I don't know is sergio here. Finally,.

B

uh Well, so not yet: okay, so cryptex is in is completed, itf less call. Now, um there's sergio's, I think working on finishing up the last changes in response to the itf. Less call comments, um so we're getting uh towards the point where.

D

B

It's it's almost done and we had an api proposal at the september 15 2020 meeting and the basic idea here is to encrypt the rtp headers on all the m sections, or none or well, it's not within a bundle or uh none of them. um So the way it works is you always attempt to negotiate cryptics.

B

So you put a cool cryptex on all the m lines in a browser that supports cryptex, and then you can set the policy whether you want to negotiate, which means you'll, accept it not being supported by the other side or require, which is that the other side has to support it or you'll fail and then on each transceiver. You get a little attribute. That tells you whether it was negotiated or not. So that's, basically what we uh what we had in the api.

B

um We had one question which was about um the uh what you know: the various uh cases where the other side doesn't support cryptex.

B

So what can happen is you a browser can send an offer to a peer that doesn't support it or, of course you can get an offer from a peer that doesn't support it um and uh it basically it cryptex is. The extension is, what's called bundle transport, um and it only requires that you have a cryptex in all the m lines of a bundle group, but you can have multiple bundle groups, uh you might not be bundling, so it doesn't require that all m lines be identical.

B

So here's an example that harold came up with where you have. Basically one bundle is for audio and the others for video, so everything's not on the same port, um and this is this would be an offer you got from, uh it would be from a non-browser, um and so here what's happening is for some reason uh the audio is doing cryptex, but the video is not uh maybe there's a different. It's a like a legacy, video system that doesn't support cryptics.

B

So, if you put in require as your rtp header encryption policy, the browser would reject this offer, but if you put in negotiate basically what will happen is the browser? Would send cryptex for audio, but not for video, so it's you can get these situations where it's sending it on some bundles, but not others, um but I think that's basically what will happen if it's negotiated and probably negotiate, has to be the default.

B

D

B

Pr now for this, which is pr106 and um I'd, encourage people to review it. uh Basically, it's here we have the header encryption policy of negotiator require uh trying to explain what that means exactly.

B

um I don't know if we need more examples with respect to this, how this works, uh but this is what's in the pr, uh it is a little complicated because uh you do have to change some of the the steps in in webrtc pc um to set the policies and stuff like that, um and then we have this. This extension, uh this attribute of the transceiver, which is the rtp header encryption, negotiated, attribute.

B

So uh what do people think about this? Any any comments.

D

Any reason it's not a get parameter, a the good perimeter.

B

uh It's uh well uh which one the encryption policy.

D

uh Previous slide.

B

Oh previously uh yeah, this would be a oh.

D

That's perfect.

B

Yeah, it's redone, it should be ready and it is good.

D

It's per transceiver.

B

Yes, okay, um yeah. It really only needs to be per bundle, but you can't can't really there's no way to really.

C

B

That so it's we put it on the transit.

F

um This looks good to me. uh I'm just wondering whether we could we, whether we want to simplify it and say with a bundle or not. We we're just reporting cryptex or not.

F

I don't know if it simplifies implementations, but it's simply playing a little bit like uh the checks uh that that we would do. It would be only in one place that you would check whether um cryptex is negotiated or not, and not for each transceiver.

B

Yeah I mean it would be because of this weird situation here right you would. uh The question is: how would you tell somebody that it was not on any of the video m lines, but it was on the audio m lines.

B

um The problem is, we have no concept of bundle and webrtc pc, so there's no there's no attribute of this bundle. It's just for each each of these mids.

G

I mean, I guess it could be a property on the transport, because that's what a bundle is.

B

Right right, right, right.

G

Yeah the map, but you know it's: what could go that way could go the other.

B

What I'd like to encourage people to do is to review it and take a look at pr106 and put in your comments. You can try to iron out any last uh things in in this.

D

The question since this is a configuration: what happens if you change it with set configuration from negotiate to require, for example, and you already have transceivers.

B

D

G

As a really bad idea, yeah.

B

It is a bad idea, but the question is: uh um do we prevent that from happening.

G

Yeah, I think I think that that's reasonable to do um yeah changing changing that is really awful and uh you know writing tests, for it is going to be another matter entirely yeah. So let's just keep it simple. Unless we see a use case, I guess.

B

I don't see it using the top of my head. There's no reason you should do that yeah. That would be. That would be horrible.

B

Yeah, okay, I think we've got as much as we can out of that one. So uh I do want to try to cover it's 27 uh 35., so it's been changing a little bit uh when he. This is what it looked like when the issue was originally filed, which is uh basically what what this is about is is some confusion. I think that may exist about rc7728.

B

And the filer basically said he was confused about what level of support is required for rfc 17. This is the rtp stream pause uh and the problem is it's. There's no level of support required, um rfc, seven, seven yeah.

G

I I can speak to this very briefly. um uh There is one place in weber tcpc uh where it specifies paying attention to the simulcast rid paused uh flag, so the little tilde that you put in uh the sdp in the simulcast attribute, so we've got one place in set remote description uh for processing remote answers that specifies looking at that paused flag from 8853 and then setting the active bit on the uh the encoding parameters to reflect uh that state.

G

But uh if you're gonna support that flag, that little tilde that requires support for 7728, and so my question was okay. If we're gonna be paying attention to that, don't we need to support 7728 and it seems like, uh like people are kind of leaning towards removing that language from webrtc pc, so just ignore that uh that little rid pause flag and that just makes the problem go away.

B

Yeah I mean my uh my question overall is why the tilt is even in there, because, um basically, it's it seems like it's a problem, because in set parameters right, you can set active or not it's it's not supposed to cause any stp renegotiation or any activity at all in sdp, um and there is something called a video layer allocation, header extension which essentially, you know sent along with the rtp packets.

B

It tells you if the samocast layer is stopped, because the bandwidth effective bandwidth is zero. So I'm not sure why the tilde is even in sdp. What what the point of that is. uh Florent.

F

H

Do you want to do you know.

B

Why this is there? What the point of this is.

H

I think that might have been an issue in the implementation.

I

H

Implementer um mixed two different features.

B

Yeah, um so I think we need to uh yeah, so I I don't I I think this probably shouldn't be. There is the answer.

B

um I think we'll probably need to uh think it over to uh decide what changes need to be made to the spec, but, based on this, I think uh we can mark 2735 as ready for pr uh and try to come up with a full request to figure out what to do.

B

Okay, so I went out since we're a little bit behind. Why don't we move skip over these uh issues uh because they're a bit complicated and move to uh capture controller anymore.

D

Yes, thank you and uh sorry for my voice. I'm a little under the weather today, so I'll go a little slowly, um all right. uh So this is yeah next slide. Please!

D

So I'm gonna propose a new api um that would cover uh two issues or maybe three um and the first one of those is conditional focus. So just to recap, uh get display.

D

Media is a tool that enables the acquisition of a user's display and that's from the spec, and that's really all it does, but all browsers today, at least the ones that allow capture of window and or tab will focus a window or tab upon capture, which assumes that the use case is traditional screen sharing in a video conference, and this is turning out to be too limiting for some apps in recent developments.

D

We have capture, handle identity and actions that exist to let apps keep the user in the vc tab and also we're hearing from some screen recording apps that it's too soon to focus the new screen, because the user hasn't hit record yet and they're asking for a way to focus later, which is not part of the proposal at the moment. But this is about providing a place where we can put potential controls and have these discussions later so where to put a focus, control api, why not the video track?

D

Well, there are two reasons: a media stream track is a media consumption abstraction that is both clonable and constrainable.

D

What that means is that you can have one source and many consumers where each track has its basically its own constraints and the constraints mechanism here lets basically arbit excuse me, will arbitrate between for some properties on a source that are not shareable. You can use constraints to negotiate, but in practice, most browsers, implement uh things like downscaling and frame destination per track. So that means that these tracks can have independent settings and that's fine for some apis, like we have new methods on the track like crop2, that is basically per clone.

D

So if you crop one track clone it doesn't affect another. However, imagine a new track focus method say would affect all clones and that's a leaky abstraction, because the user's focus is not an inherent property of a single video track, but of the source and the other problem with putting it on the video track. Is that why only the video track? What about the audio track? Do they both have this property? If so, why? Why not?

D

And it's confusing so the problem here is that we've stepped out of media consumption to remote control and we need. I think we need a higher level object to do that, to put that these controls on next slide.

D

Yeah and so there's a similar issue with um uh capture handle actions, um there's a method to send the captur action uh to these.

D

It's an api to send supported actions to the captured page that we're we're still uh discussing it's still early times for it, but it also seems misplaced on the media stream track where it is right now, and the reasons are the same. uh Basically, one source, many consumers and, unlike crop2, send capital action would affect all clones again. A leaky abstraction because progression of the captured page isn't the property of a single video track and same thing. I will disappear both on the audio and video track. Who knows so?

D

We've stepped out of media consumption to remote control. Here, a higher level control object seems needed that is owned by the caller of get to fly, media isn't cloneable or necessarily shared with every consumer. Next slide.

D

And similarly, uh the same argument can be applied to get capture handle uh the spec already know that there's no consensus yet on whether get capture handle belongs to on media stream track or on a dedicated controller object. That is neither clonable nor transferable.

D

However, chrome has already shipped this, so this proposal focuses primarily on conditional focus and actions at the moment, even though identity isn't mentioned in the slides, I have coming up next line.

D

So this is the idea. um This uses a pattern, that's familiar from those who used a board controller with fetch. You first create a new capture, controller object, and then you pass it in as an argument to get display media what this does. It does a couple of things it upon success.

D

This controller is associated one-on-one with this get display. Media call- and this is a new advantage because, unlike uh say media devices, if you get an event on media devices, you don't know which uh you know which specific capture you can have multiple captures going on. At the same time, you know which you don't know which one it is, but here you have a dedicated object to this particular capture, so you could do things like if we wanted to later- or we could put.

D

This could solve identity where you could put origin and handle directly on this controller object, and you could have a focus method on the controller that returns a promise. So the idea here is that if you don't specify a controller, you get backwards, compatible behavior, that the browser will automatically focus in window or tab. But if you specify a controller, it will not automatically focus unless the application calls focus, and this could be the the application could call this even before calling get display media or you could call it shortly after and in the past.

D

We've had really hard time defining for how long after there are some security issues there potentially, but we also have apps that are asked for this.

D

So maybe this would open the door to discuss potential future changes there, but I don't see a problem, for example, of waiting allowing up to one second after after the call seems fine, as long as the end user can associate the focus with their action, and that seems similar to transient activation, for example, the problem we used to have was correlating this focused method with a single get display media success, and we were talking about uh within the certain within the next task and really sort of tighten it uh control there.

D

I don't believe- and this this kind of solves that, because we have a dedicated controller object for this capture only and same for actions. We could also put get supported actions on this controller and if, if they are supported, uh you would get a non-empty array, otherwise you'll get an empty array and you can send an action on your using your controller dot. Send action next slide, for example, and that's it.

E

We have a lab in the queue.

A

Yes, hello, uh so I would need to think about this a bit more, but generally the controller uh pattern seems nice to me. uh If this helps us move forward with all sorts of um proposals, all the better um a couple of questions uh would you be open to exposing a getter for the controller on a track, or would you consider that this actually goes against the grain of your proposal and that's number one and number two? What about a proposed new api right now? That returns several streams?

A

Would you also want to return several controllers or, or would you say like okay, it returns several streams, but each one has a getter for a controller.

D

um Sure so, on the first question, I think uh the purpose here would be to not ex it would be a feature to not expose it on the track. So I would discourage having a method to get to it, because the goal here was to return an object that the owner, the caller of get display media, can keep to themselves and and not necessarily distribute to every downstream media consumer that they may have for clones and that kind of stuff.

D

Because again, if you have a single track, that might be one clone of many. And uh it's not clear to me that if a javascript application wanted this, they could easily add this controller as a custom attribute to their track. Right.

A

That's one thing they could do, but what another thing that you could do is you could say that um when you clone or when you transfer, you don't actually transfer the controller and that you do separately but yeah. I guess that you do make a good point that if somebody really wants to attach it to the track, they can do the it themselves.

A

So um yeah good point.

D

Okay, great and the second question uh yeah, I haven't really put anything about multi capture here and I I'm open to discuss that. If we think that that's useful, um I guess I haven't thought through how you know it seems like uh intuitively. I would have expected one controller per per capture, but if you can think of use cases for having a shared one, we could also entertain that. I guess.

A

um I would say that if you're capturing several things at once with an api that is not currently standardized but has been proposed, and then you might actually have completely different actions for the different uh things uh and I was kind of leaning towards a model in which you different, depending on what you capture. If it's a window, if it's a tab, if it's a screen, you might have different a different um subclass of ministering track and different things exposed.

A

It looks like a controller, would have a bit more trouble doing that unless you also subclass the controller and gave it different properties, because imagine like focus, for example, makes sense for capturing a window or a tab, but probably not for capturing a screen. But I imagine you would still want to expose focus regardless so and then what it would raise. An exception.

D

Yeah, so so I put a little um disclaimer at the bottom here. So what did I say here, sort of my speaker notes down there um um so yeah. The idea of focus would be that it would return for a captured window tab. It would resolve when the tab window got focused, but no earlier than get media get display. Media.

F

D

But for a monitor, it would just basically resolve immediately upon get display media success. I didn't see, I mean make it a not. Basically, we could also reject if, if you prefer.

A

So what I would have liked with subclassing is that you know the application will just be able to check like hey. Do I have a focus method and screens would simply not have that.

A

Right, well, I mean also audio tracks, but that's regardless of your control, right.

D

So yeah so mostly.

A

About you know the difference between something that is focusable and something that is not.

D

Right so the goal is to get an owner object for your capture, and given that there can be different kinds of capture I mean we could always discuss making different subclasses of capture controller. I think that's a bit uh excessive at this point, but you know I think we can always.

D

E

Are you running you're next.

F

um So I I think that the the idea to separate uh uh functionalities that are specific to a source and not to a track uh is a good idea. So we we should do that. We try to add that to track and it's uh it's not a great model, so have it elsewhere, where there's a single object uh is, is a good idea.

F

um I I'm not not sure about the api surface, but we can probably uh iterate on it. For instance, focus is one thing supported. Actions is another set of things, and maybe it's not just one object, but it might be two objects.

F

For instance, you might want to have a transferable action controller because you might want the track to be transferred to a worker, and then you might want to control actions in the same place, for instance, but transferring focus handling does not make sense, so I think we should.

F

uh We should dig into that uh apply this idea of separating these functionalities out of track and see where it should be put in the same object in a different object in a controller kind of style or in an object that is created by the display itself and so on and yeah. Let's continue uh discussing it and improving it.

A

So um I would have a quick question: um how quickly do we think if we all agree, assuming that we agree that we want to go this way? How quickly can we actually agree that this is the shape and you know get it done, because otherwise it could be that now other discussions that have started a very long time ago, for example, conditional focus, would now get you know scheduled behind. Yet more discussions.

D

Right, yes, so I I would say that um I'm not a huge fan of subclassing and that it has some benefits. I understand, for you know a method: it's not there, but that's usually just going to throw in a different place. You're going to get an you know, type error instead of can't focus, error or whatever, so it doesn't seem that different to me, um I think the main benefit here is to have an object where we don't have one today, um and I would certainly welcome getting this ready.

D

For um I understand, um conditional focus is uh probably the the most immediately useful. uh So I think we hopefully can go this direction for that. So we don't end up.

D

You can already detect what type of capture you've received right.

A

In theory, in practice, only chrome actually implements that.

D

Yes, but uh also no one implements this yet so we could definitely implement uh uh both of those I mean it seems like. If you have a capture controller object, it'd be very useful to know what one useful property would be, what what type of source do. I have- and I don't see that, but.

A

We do have that already a quick question: uh why do you mean? What do you mean that focus would throw elsewhere if it doesn't like? If we subclass, I mean, I would expect that the developer would check, if not not, focus right before calling focus if they know that this doesn't necessarily have focus the focus method.

D

Sure, but the the benefits of subclassing that you get from c, plus plus, for example, where things don't compile you catch it earlier are not present in javascript, so you'll you'll get javascript developers that just call focus because it worked when they tried it and something returns, a different type, suddenly user picks a different type and then focus is now not a function of undefined as the error they would get right. So I'm saying supplies doesn't bias that much in javascript.

F

Yeah, I would also try to avoid surprising and use compressibility like we can use different objects or different double checks of a main object, or things like that to group things properly.

E

So to to to a large point and assuming that conditional focus is the one topic where we have the greatest clarity, maybe capture controller should focus on so to speak. uh Let's should focus on on this and when the time comes to look at supported actions, we can look whether they share and offer the same properties across the two needs.

E

That way we don't have to solve all the issues we can make.

D

E

Make this pattern work for focus and then adjust or create new classes as needed that works for me.

F

Yeah sounds good.

A

uh If there is time for one more quick question.

A

Okay, nobody objects. uh What what do we think this means for backwards compatibility I mean for if capture controller does not exist on the platform, so uh controller equals new capture. Controller is probably going to throw, whereas you know the yes, so that looks less than the best solution. Hopefully there is a better solution.

F

Capture controller and it's easier to feature detect an interface than feature detect. Whether a field in a dictionary will have an effect or is supported.

E

Yeah, for better or for worse, that's how most apis are being deployed, doesn't make for very good looking code, but but it allows for this kind of evolution.

A

D

A

So to me, it's not 100 clear if this is better than putting things on tracks, but if this helps us move forward faster and with more consensus, then I would consider that a huge plus.

A

E

My statement um so concretely, if we do focus on focus, uh I guess this makes it a screen share, recall proposal rather than a capture handle where it's right now being discussed.

E

I'm trying to make sure we have a clear place for iterating on the proposal.

D

I don't know that we have a clear place, but we could certainly put it in screen capture that.

F

Makes sense to me we.

D

Anticipate okay.

D

No other questions.

E

You always what an issue pull request.

D

I can do that yeah.

E

Which one I'm sorry, which one an issue or a pull request.

D

I can provide either or both.

F

Okay, I would I would say: issue is good so that we can talk a little bit about the the api surface first in the issue and then in a pull request sounds good.

D

I'll do that thanks.

E

uh Bernard, I think we can move on.

E

Then I have to.

B

Right but still uh there we go all right.

B

Okay, so wherever she encoded transform, you went.

F

Yeah so for about, we were there a couple of meetings ago, uh we agreed to add, generate keyframe uh to rtp script transformer so that uh when you're, adding a transform, a webrtc and kodi transform you're able to control the encoder and ask the encoder to generate a keyframe and currently that's working great.

F

You can even pass some parameters to say in case of simulcast. I want keyframes for that and that encoder and that's working fine, it's an async method. So it's returning a promise and the promise is resolving at the time the first keyframe is enqueued in the stream.

F

So when the promise is resolved, you know that you will be able to read and get to the keyframe very very quickly. It was mentioned uh during uh the generation of the pr and uh discussions that, uh if the promise could return the timestamp, the keyframe timestamp, it would be a nice addition and uh that's true. It would make life a little bit easier. Probably, um but since generate keyframe is able currently to generate several keyframes from different encoders.

F

We would need to generate multiple timestamps and potentially resolve at the last uh timestamp. So it does not work great with multiple reads so next slide. The idea is to basically uh remove uh so to change the parameter from a sequence, just one parameter, so the old version is uh returning a promise with no value, and it has a sequence and the new version would be uh returning a promise that would result to timestamp and uh there will be an optional read, meaning that you can select which anchor encoder you you want.

F

The old api can be shimmed with a new version using promise race. So that's that's, probably fine and if no read is provided because it's an external parameter, then the first hand color will be selected.

F

So that's the proposal and there's a corresponding pr 132 about that fruits.

B

I think it's okay.

B

F

So if there's a agreement, we can go to the next topic and mark and then editors will be able to finalize upr. Okay um now going to media capture extensions, so we discussed.

F

In the past background blur- and uh we know that background blur is being deployed in web pages a lot these days, a lot of websites are actually using canvas, webgl, whatever web technology to implement background blur, and it's working, it's also being supported by os's like ios and macos, for instance, have bank and blur built-in support and uh riju and hero did some measurements.

F

I believe, and they can talk about the measurements if the perf measurements are more, but basically it might be uh two times or even more power efficient to use the os background blur than the web technician background blur and also, if background blur is already applied by the os, it does not make any sense for the website to apply background blur on top of it.

F

So I think this explains why it's useful to expose support of wet background blur to camera tracks so next slide.

F

The proposal is to add a background blur constraint. uh It will be, it would be a capability and a setting as well, and it will allow applications to identify whether background blur is supported by by vos, and uh it would also allow the application to identify whether a given track is already blurred or not by the os and, of course, with apply constraints. You would be able to switch on and off background blur.

F

So the proposal is also to make background blur a boolean constraint for now, uh oss are more or less like echo cancellation, it's uh enabled or not, and uh we could start with this. um I think that we we can evaluate the usefulness for a zero one kind of double constraint, and uh if we go there, we would need to understand how to define it.

F

What means zero, probably uh number, one blur what means one, probably a full background blur, but what is full by gun blur, so we will need to to to to work on that. So that's why. Currently, um um the proposal is to start with a simple, brilliant background blur and um I like to get feedback on that.

F

uh If the working group thinks it's uh it's a good approach and if the working group thinks that uh if we switch to double at some point, what would be the cost and uh should we if we want to switch to the ball, should we switch before shipping uh this thing and and so on so fruits.

D

Looks good to me.

I

Okay, so just to be clear, we can shift to a double later on right I mean we can think of that, because all the frameworks are having the double, but the native platforms are having boolean right now, but things might change and we need to adapt accordingly.

F

Yeah it's I I'm not exactly sure about the migration path. One migration would be to to add a new constraint which should be a double, for instance, maybe there's, uh maybe when we'll implement it at that point, all acs will have changed to a double and there will be a clear definition. So we we can switch to the ball before then, but yeah, starting with boolean, is probably the safe seems like a safe approach to me. Right now,.

F

Okay, okay, so I guess uh it's already okay, so uh we can probably uh work on uh merging the pr and finalizing the pr uh for editors. um So next slide.

F

The second part of the pr uh the same pr is the introduction of a new event called configuration change, and the reason is that some oasis may change camera microphone, settings outside of user agent control, for instance, uh you're you're in safari or you're in an application, and uh you grab camera access and at some point the user might decide to enable through the os ui a background blur.

F

So initially the stream was not background blurred by the os and after that, it's background by the os and some oss will not allow web applications to to switch it off. But in any case, there's a change of settings. It went from false to true and web applications might want to be notified of that, and currently the only way would be to basically call get settings over and over and over. So the proposal here is to add, like a configure exchange event, so it will tell the web application.

F

Hey you, if you, if you're interested in understanding, what's what's happened, please call get settings, get capabilities and look at whether these things are good for you. So it would be a simple event. We would start with a simple event like no fields, no, nothing and there's a small example there that says hey. If, uh whenever configuration is changing, we will we will toggle back on blur enough if we, if we, if we can uh dom you're on the queue.

E

Yes, that configuration change event would apply to any constraints, to subset of constraints to only background blur. What's your model.

F

So yeah I forgot to mention that it would be on the track on the track object and uh it would apply to any settings capabilities. uh We can, but that's my understanding. We could restrict it to only some set some uh some values, but um we should. We would need to have a good reason for that and I feel like it might be useful in the future for other properties as well.

D

So um this made me think about uh previous slide, um so this is um apply constraints on each track, clone right. So how do you deal with um if this is a global thing in the os? Only how do you, I guess, you're, not supporting exact, for example,.

F

um So so, if in ios, if vos is, if you enable background blur through vos, then the web application cannot do anything. So the idea would be for get capabilities of background blur to go from false to to just true. And then, if you try to do a byte constraint by background by force, it would reject.

D

It it would reject how.

F

Yeah, because the background blur capabilities would would only be true, it would be a change, the os would say now it's only true and it cannot be forced to it cannot be disabled. So that's why.

G

It's a change in.

F

Either settings or capabilities.

D

But if you don't have exact constraints, you can't get rejection right.

F

uh Yeah right right, yeah, then, uh if you, if you try to apply it with a non-exact constraint, then it would. It would stick to you too. Basically, even if you set it to false, as idea.

D

And I suppose, um if a user agent for, for instance, wanted to implement this in the browser, they could do this per camera, or they could do this per track. Basically, if they wanted to, that would be okay. Okay, thank you.

F

I I would guess so: yeah yeah.

F

So that's uh that's a that's a good yeah! That's a good thing to to to discuss. um For instance, uh I know that on ios, if you disable a cancellation in safari you we would not be able to to only uh disable it for one track. So there are there precedence for things that are leaking already. If you apply constraints on one track and not the other.

D

So uh one more thing, a side effect of this is that applications can control turn this on and off with the block constraints. Is that something you had in mind or.

F

um So to to me, if the os is uh is not applying background blur, then it it would be nice to to allow the browser to enable it and disable it. For instance, that's something we could do uh at the browser convenience.

F

But in any case, constraints are flexible enough, so that browsers can just set the capabilities to true to false or to too false.

F

H

You're on the queue, yes, I was wondering what other settings and capabilities would trigger the configuration change you mentioned always level ones, which ones are they and would it apply? Also if the same device was opened by multiple pages and one of them was, for example, changing the brightness of the exposure mode and different things like that.

F

Yeah, so that's the same thing with um with grenade, for instance, in macos. If two applications are capturing, then they might compete with camera settings and uh what we try in the browser world is usually to uh to hide this, and we can sort of doing doing that and we can continue doing that it will. The configuration change event uh will be at the discretion of uh of the user agent. uh On that.

F

uh But if two pages are capturing, I would guess that uh if the user agent is in full control there, so it can do whatever it wants.

H

Yes, it would be easy even to broadcast messages to another page in the same browser, capturing.

F

Okay and it's it's true that there might be like some uh security or uh privacy issues like if you're capturing between two different origins, uh so you need to be cautious and in the pr we probably need to uh to add some warnings there uh in safari. I don't think that it would be possible, because there's only one page that can capture at any point in time, and so, if the track is, for instance, muted, we should probably not fire the configuration change event. Even if a configuration has changed yeah.

F

You never mentioned it as well time fuzzing like for device change.

H

Yes, because it could be used also as a covert communication channel yep.

F

So if you, if you can add the comments on the pr forum and universe, did maybe so that would be good so that we can make progress.

F

Okay, so I think that for pr, 61 there's uh uh consensus to move forward, provided we improved, maybe the pr in terms of the things we discussed and privacy and so on- is that correct.

D

Rough agreement, at least yes,.

B

F

So let's go to the next slide, then.

F

So, henrik from google uh uploaded a pr which is about adding poor, efficient pixel format constraints, so it was pretty good to describe it there before uh further uh making progress on vpr. uh But the issue is that some cameras generate motion. Jpeg video frames, especially uh for some like with eight frame rate, combinations and uh os, will typically decompress.

F

uh These video frames before feeding feeding them to the user agent, which usually expect like yuv or like uh non-compressed uh video frames, uh and if there is a hardware decoding, then the impact of actually capturing motion. Jpeg is okay, but if there's, if mobility decoding is expensive and for some machine, it is then there's a potential power impact. And uh apparently google can made some measurements and validated value that it was an issue and some applications would like to be able to avoid um these configurations so avoid uh the motion.

F

Gpek decoding penalty, so the proposal there is to add a boolean power, efficient, pixel format, constraints that would be exposed in capabilities and settings and it would be uh you could use it in when calling get to the media or apply constraints as well.

F

D

So this would return true always except for cameras with motion jpeg and no hardware.

F

um Yeah with the right, with eight frame rate, combinations that uh um that forbid to not that uh force you to actually use motion, gpeg format, yeah.

F

Yeah, I like it and probably uh it's uh another small fingerprinting issue, meaning that you would be able to detect that some cameras are supporting motion jpeg. But since it's after get user media being granted, I think it's fine.

E

So I'm trying to understand.

E

So it looks like a fix me constraint like: when: would you want to have a power, inefficient, pixel format.

F

If you, if you require a specific night, for instance, like I don't know when motion jpeg, video frame is being used, but let's say it's uh for super hd uh resolutions, and you actually want to capture super hd resolutions because you're capturing one you just want to capture a single frame, for instance, and nothing more in that case, maybe using motion jpeg, you don't care about it for typical video conferences uh system. I would guess uh you would like to enable it.

F

The issue is that so the default value would still be to allow power in efficient pixel formats, because we're we're not sure about uh potential uh backup compatibility issues.

E

Now I guess I was expecting everyone we using get user media for webrtc to update code to add that constraint and that feels.

E

F

Yeah, I would, I would be happy to reverse the default so that by default we would uh not use uh the power efficient. We will not use inefficient pixel formats. um My guess is that there's still leeway for the user agent to actually.

F

Not select motion gpeg if it can, but if it cannot like, if you're setting with a to exact constraints, for instance, and there's only motion jpeg, then you would still select it and and yeah. That's that's fine, but I would hope that user agents in general will try to favor not selecting over efficient pixel formats.

E

But I guess that's a bit what I mean like if you're asking for an exact resolution, then, as you say, probably you don't care about the power. You understand that you're asking for something weird and specific. If I'm just you know here doing my get your media so that I can send it over wherever you see, I really don't want to have to tell you to do that efficiently. Just do it!

E

F

So sure so the the point is also: let's say you, you ask for a 640 480 and you actually get get uh 1024 to avoid motion jpeg. Then it's good that you understand through settings and capabilities that the user just tried to avoid uh avoid it due to motion jpeg and.

D

F

If you really prefer at that point to use 640 or maybe another kind of a resolution, then you might be able to do it with the knowledge that it was to avoid motion, jpeg and currently web applications. Do not have that.

D

Knowledge and yeah, and also I think, um user agents might have different defaults for this. Maybe for desktop it's not a big deal to use motion jpeg, maybe on mobile phones. uh It would be.

D

So I don't know that everyone is. I don't know that we know that everyone wants power, efficient, pixel format, always because I think uh it does.

A

D

Constraints on no pun intended uh what you can get as far as resolutions and stuff. So I think the main advantage here is that you can get a resolution, a high resolution you want and then you check get settings. Is it also power efficient like oh? No, it's not. I'm gonna make different compromises.

F

Yeah yeah, if, if we were to restart from scratch, I would probably uh try to make this power efficient, pixel format on by uh on by default, but it's it's probably too late. So I think it's fine like this.

E

I mean I I guess. As long as user agents can already do the right thing, uh then I would be trying with exposing the information I I just like. If that makes it so that I don't say: oh it's not my responsibility, then I think it's really the worst possible outcome.

D

Because we know.

E

That, maybe 0.001 of developers will know about this and will know how to apply it correctly across desktop and mobile because, as you never said, maybe as a developer, you would also need to make that distinction. So I okay, the gate, settings uk is, I think, is fine. The device selection.

F

So for device selection, I totally agree the apply constraints thing it's fine as well like. If you there's no way you can discover currently- and that's really sad uh like the different uh part of the camera and that's something I I think we should we should do, but in any case uh having this constraint with it available for apply constraint is good for device selection.

F

I don't think it makes sense at all, but that's why our efficient pixel format will not be able to be a required constraint if you uh colgate user media, like all the constraints, cannot be used as required constraints, forget user media and that's that's a good thing and we would follow this rule for efficient pixel format.

D

Yeah, I agree with that. I think uh limiting some users may only have motion generate cameras and we don't want to exclude them from using a website.

E

So yeah, I guess we I'm carrying this is worth doing.

F

Okay sounds good, then um we will mention that in the pr and proceed uh with uh finally finalizing.

D

F

Prt adjustment.

A

Hello, um thank you. So I would like to talk about sharing and specifically sharing tabs, but actually we can generalize the image to sharing anything. Usually when a user uh gets ready to share a presentation or to explain things to people, they might have several surfaces that they might want to switch between. For example, I might have you know something for editing code, mdn wikipedia stack overflow, all of them ready, scrolled to just the right place, and I'm just about to start uh giving the presentation I share.

A

I show the first thing: everything's great people can't stop themselves from uploading and then I need to start sharing the next thing, and here it gets a bit tricky next slide. Please, because how do I even go about starting to share something else? If I'm not using google me, I need to go to the tab for the feeder conference right. So I need to find that out of potentially many tabs and many windows, then I need to trigger a cool another call to get display media right.

A

I get a list of tabs and I need to find the new tab and I might actually have to do that a couple of times before I get the right one and then I try to find my train of thought again.

A

I try to start keep presenting and then I need to switch surfaces yet again and it goes on and on so obviously not ideal next slide, please so there's an obvious solution and the solution is that right now, chrome in the uh using an extension api allows you to just go to the tab that you want to share instead and press a little button saying share this tab.

A

Instead, obviously, we would like to ship that for absolutely all websites out there, but that's a little bit problematic next slide, please, because until now this did not exist. Many applications are built with the assumption that it does not exist if they get something and especially if they establish a connection using capture handle, but they can do it using other means they might start relying on that right. They could expose controls.

A

They might rely on the fact that they're self-capturing, all sorts of things, uh could be uh assumed that are correct initially, but then, if the user presses share, this type instead would no longer work next slide base.

A

So here we've got one example. So currently, if you're using google slides or google docs, you could embed the meat inside and it kind of assumes that you're capturing the current tab. If you try to capture any other tab, it tells you hey. What are you doing and it shows you an error um sure it could also transmit that other tab remotely, but that's a product decision right. That's not a decision for user agent.

A

If, for some reason the application does not think that the user should be able to choose anything else, so be it, and if the user has already chosen the right thing and all sorts of nice things are happening and the user is happy. We don't want to confuse the user by showing him a button which once pressed breaks everything down next slide. Please.

A

Similarly, let's say that we've got two tabs going and one of them is capturing a slides deck and we actually uh remote control the tab and then maybe we switch to capturing docs or maybe we started capturing docs and we can actually scroll in the middle and that scrolls the capture, tab too, the user might think hey, that's just the general thing that can happen right like if I capture another tab. I can scroll it from this tab. Then they try to share another tab. Doesn't work and they're upset? But you know next slide page.

A

And basically, the underlying issue here is that when you show users big, shiny buttons, some of them end up pressing those buttons- sometimes that's great, sometimes not so much, and the application would be the one that gets the blame right, because not all users are going to understand the difference between browser application.

A

The application could not have controlled that, basically, all of those webby uh pet peeves that we have with web applications is because they're kind of limited in what they can do in comparison with native applications, because they don't fully control the uh the experience and when we can give them a bit more of control, we can make them a bit more polished.

A

So next slide. Please. I suggest that the uh the solution here is mind-bogglingly simple. All we need to do is yes. All we need to do is just expose an extra boolean. The application can tell us if it's interested in dynamic sources and then it's up to the browser to decide what dynamic sources it wants, how it exposes it all of the normal things, but it could even disregard what the application says or not that's up for discussion, but at least the application.

A

The user agent is going to be in a position to make an informed decision and next slide. Please discussion.

E

F

um Yeah, I'm a bit surprised by this uh api surface. um We need to think a bit more about the use case and so on, because it seems that maybe like for the self capture thing it's uh the user agent can already do some heuristics. So that's one thing, but in any case um I'm not sure that the web application can decide whether it wants uh the search change, supported to be true or false. At the time it calls get display media.

F

Maybe it will want uh the actual track like oh, is it a self-capture or is it oh, it's not self-capture and so on uh to actually decide whether it wants to be able to change source. So maybe there should maybe there's a I. I wonder whether you you had discussions for a more flexible api surface or not or it if this a particular api surfaces.

F

I just want to focus on one simple case and not not extend it to anything.

A

uh So, thank you. You've mentioned two things uh going now in the reverse direct uh order. uh If you want to make the application give the application even more uh power to make it a later make an even more informed decision, uh I'm up for it, and we can discuss exactly how to do that when you mentioned heuristics, which was the first thing, I would like to push back on that. I think that heuristics are going to serve some applications, but not others.

A

So, basically, so as there are two applications out there that want an opposite heuristic, they're gonna, one of them is gonna, be unhappy, and I would like to keep everybody happy and they think that we could.

F

So what would be the default, then? That's.

A

That we can discuss that would be uh we could discuss. I would argue that, uh because we don't want to break any or even inconvenience existing applications we can just uh default to the existing behavior of sources are not dynamic, but I'm open to discussion, because the important thing thing, I think, is to allow the application to tell us what it wants.

D

So, um yes, um I'm a bit concerned about this api because it would let the application limit the user's choices, and I, like the feature in chrome that you can switch. I think dynamic. Switching is um a good idea, but it also has some raises some questions like if the user changes source is that another opportunity will it focus the new source? Will it not focus it?

D

uh It's not clear to me how that would work, and also I tried this in, um and this might be good for google meet to to responsibly set this, but this would be for any web application and they could turn off this feature on users and that doesn't seem as desirable as the having this decision in the user agent. So I tried.

D

Google, google docs has the integrated meat feature now in chrome, and I tried it, and so I'm wondering if this is actually um related to that and that the the problem there seems to be to keep the user from picking another source which, in the current chrome, is a problem already. When I start presenting, I'm actually get the chrome picker that defaults to this tab, but there's still other options that say other tabs, and if I pick some of those I get the you mentioned, uh why do we click this button?

D

I click this button and google me google docs now tells me sorry. You can't do that. You have to pick the same tab. So I'm wondering if this is a subcategory. If this is if the problem, this is really solving. Is we want self capture here and we have a specific use case for soft capture, so maybe the user agent should just determine this based on whether this is self-capture or not, because self-capture seems very different from capturing any source.

D

I can't see what kind of app would break um in the general case of not picking self-capture, for instance capture you might still have.

D

You might have actions, for example, or uh identity capture handle identity, um but you could solve that the same way you would solve it for navigation, for example.

D

So there's really no inherent difference to me between capturing a another tab and navigating doc from document a to b in that tab versus um picking a different tab b that happens to have document b in it.

D

As far as games are exposed to javascript, it seems like that should be possible and we don't need to turn off this source change, at least for uh capture, handle, identity and actions. Unless it's soft capture.

A

Okay, um you've said a couple of things and I uh I hope that I'll be able to respond to all, and if I forget one of them, please remind me. The first thing that you uh said is that this would allow us to limit user selection.

D

A

I think that you should look at it the other way around right now there is no user choice. You cannot actually press share this tab instead, so this is a boolean and if we look at it as we can look at it as actually introducing new behavior, so we're not limiting the user, we're actually giving him uh and the new choice responsibly.

A

um Second, is that um you didn't actually say it exactly at this time, but I think that we've discussed this before you were, if I'm not mistaken, concerned that this could be used in order to kind of nudge, the user towards certain things, and I would just argue uh here publicly that this could not happen anymore than before, because basically, when the user chooses to select something uh is the moment where you would push him.

A

You've mentioned how uh google docs uses uh prefer current tab, and yes, that's true, but that's unrelated to this particular thing that happens before we even show the share this tab instead button.

A

um Another thing that you said was about focus and you.

C

A

This would handle, and at least with the chrome implementation of this ux you are. Actually you must be focused on the new tab at the moment where you press this. So it's no issue, it could be that you are gonna, choose a different ux and then maybe you will want to standardize a bit more something about what does with focus and I'm sure that we'll uh be able to find a good compromise there, um and the fourth thing that you said was that it seems to you like.

A

This is specific for self-capture, and I will admit that this is very uh so self-capture is the easiest example of when this is interesting, and then I would go back to what you end said. It's like okay, why not use a heuristic? I would claim that if you've got two different uh applications, let's say that google docs wants one behavior and uh microsoft office wants another. You wouldn't want different browsers to behave differently. You would, you know, employ different heuristics. You wouldn't want one application to.

A

You know to be ill-served just because the other one was there first or is louder or whatever. It makes sense. For this to actually cater to the specific application, I cannot come up with a heuristic that we would fit all of them. Otherwise we would just hard code that behavior and the last thing that you said um I forgot that one. Could you remind me.

D

um I'm not sure that um well I I didn't actually suggest that this could be used to, of course, the user. So uh um I I think my main comment here is that uh it'd be useful to see if user agents- I oh you mentioned um that this is a way to enable features, but I think it would be better to leave the application out of that decision and I think it's fine neces. It doesn't mean that this seems early to stand wise to me, because this is still a user agent space.

D

In my opinion, where it's good, that uh browser vendors are experimenting with new features in this area, and I would hope that they could do that and maybe make a special case for self-capture it doesn't seem. I don't understand why letting all applications control this feature is necessary.

A

Oh okay, I will respond to this, but I did actually recall now the fifth thing. The fifth thing that you mentioned was navigation and you asked why this is not the same and uh yes and by the way. Sorry, if I put words in your mouth about queers in the user, I thought that you said something to that effect in the editors meeting, but maybe I misremembered so um you're saying it's good. That user agents are experimenting with this and uh I respond to that that we've got implications even only inside of google.

A

We've already got applications that are pulling in different directions, one of them. What wants to have share this tab instead and one doesn't and the heuristic does not actually allow us to uh to give them different behavior, but even if it did what happens if microsoft comes next and says that the heuristic that we employ is exactly the inverse of what they would like to do. How do we make uh a fair decision there?

A

So it seems to me that there is enough web development web developer interest in this, that we need to actually provide this, but on the other hand, we cannot provide this without getting some more information from each application, and I think that unless it's actually dangerous to expose or to not explode this, we should actually allow the application to influence this.

A

Is it okay? If I also respond to navigation before handing the mic back to you aren't sure? Okay, so at least for self capture. Navigation is interesting in that it just stops the capture. So it's a non-issue there and when it comes for navigating a capture tab, you are correct that okay, so the user can do that and break things. But there are some problems that we cannot solve today. It doesn't mean that we should not try to solve other problems. So that is my answer. Navigation.

A

It is pretty clear that we're not going to stop the user from navigating a capture tab, but allowing or not allowing a share this tab instead button that is within our power to affect.

F

um Quick question as well source change supported.

F

I would guess it's just a hint right and there will be no hard requirements in any case because it's it's all ui in the chrome. So it's just the web application would prefer the user agent to actually not show a button, but then the user agent is able to do whatever it wants. Correct.

A

I am completely open to go in one way or the other, as the consensus goes, so it can be a hint or it can be a requirement. It is not important for me because, uh basically, because in chrome we intend to just abide by the hint, but if you would like it to be a hint so be it.

D

uh Anymore, well, um I'm also not sure um that the effects of swords change regardless of this api, one of the effects of chrome changing its source on the application. Today um I mean constraints are going to update.

D

um I guess you don't support label, so it's not a big deal, but then I'm wondering if this is the all you would need for this use case uh like are we gonna get prs to add events that fire, for example,.

A

um Not at the moment, but we can discuss that right now uh I would say I.

A

So long as you only change between different tabs, I don't think that control constraints need to change, and that is our current implementation. Ideally one day we would want to support. You know changing between more dynamically between more things, but that's you know for the future, uh and then we can tackle the challenges then, but right now you sure want that you start sharing another. The most that can happen is that resolution changes all of a sudden, but that can also happen if you just resize the the window that contains the tab.

A

So it's not much of a change there.

F

It could be a configuration change event as well, which we really think that's useful.

D

Right, so uh I'm not supportive of this, but uh as an interest of progress. The I'd like to at least like said on the default here, it seems like requiring applications to provide this before you get a source change in one browser. Is it would be better to flip it and say it sounds like the constraint here is applications that don't want a source change and the default should be benefits for the user, so maybe like a prevent source change, at least that's.

A

Okay for me, yes, if you would like that, that's okay for me, but just so I could understand you're, saying you're, not supportive. Does it mean that you're not supportive but you'll, say but you'll not block it or are you saying you're, not supportive, you're gonna block this, but just in case your mind is changed in the future. Let's bike should so, which interpretation is it well.

D

I'm not convinced that the user agent couldn't figure out uh eurasic here on his own, that would work for both chrome and other uh and yeah and uh microsoft. For example. I mean it seems to me that the question here is that source change is harmful on self-capture because it violates the assumptions- and this is much clearer once we have get viewport media, because I assume that's currently written. This will display. Media constraints is also reused by get viewport media, so I would object to source change on that.

A

We can say that this uh has no effect on get viewport media if you're concerned about that and about microsoft versus uh google et cetera. Well, what happens if we decide okay source uh self capture? You don't get the share this type. Instead button then loom suddenly comes over and says, like hey users often start recording the current tab, then they switch and start recording another. Why have you made that impossible? For us? This is exactly what we're looking for.

A

I don't know if phloem is interested in that I'm not in contact with them, but just you know any other kind of application. What would you say to them when I then say hey mozilla told us go with big tech and do what they like. You know. That's.

F

I I think that hint is uh is, is a way for both chrome and uh to move forward and for others to try to do heuristics and a good default, which would be uh that the the in chrome, the icon will be shown seems like also uh something we maybe can get consensus on as well.

A

um I agree: I think that the hint uh would be an easy compromise. What do you say anywhere.

D

Well, so then it would have to be avoid source change. Then I guess or not prevent or something.

A

We can do that.

D

And it would be limited to get display media and not get viewport media. That works for me too, which makes me wonder if you so sort of a workaround today for the lack of get viewport media is that you pick your own tab in the picker today. So it's almost like, it should default to often.

D

And then maybe you would specify it with false. I don't know.

F

Yeah, but it might be a good heuristic, and uh this can, if we see that it's a good, uh a good default, then we can either standardize it later on or we can like. These agents will still be able to implement it, no matter what.

A

I mean in the interest of progress, I'm okay with any default value that you would like, or no default value too. If you think that it's better, if each implementation does its own thing,.

D

All right, it sounds like we have something we can iterate on.

A

um Just so I'm clear, are you saying yes? Can I always say it's a hint we're gonna bike should do the name, we're gonna bike shed on default value, but um essentially we're in agreement here.

F

Dummy is on the queue I don't know if.

A

E

um Yeah, I guess the time has gone. I mean I, I think, I'd like to understand better the relationship. Indeed with self-capture I mean I I I take your point ela that may maybe apps that would need this beyond self-capture, but then maybe we can discuss a broader thing once we have these apps knocking out order, but if there is this consensus emerging around the hint, that sounds like a pretty good approach to me. So.

D

Well, I think it would be good in parallel to to if you could produce a use case, that is not self-capture.

A

um I have produced that. I know that uh I acknowledge that it's a little bit less convincing than self-capture, but it is what, if right now, I capture slides in a different tab and they start remote controlling that, and then I find out that they cannot actually remote control other things and let's say that the entire experience is about you know being able to present slides nothing else. Right. Let's say it's a new video conferencing tool. That's built only around that! It's not the most convincing example, but it is possible.

E

But like the heuristic here could be once you start using capture handle then no longer shows a button, because you think it's going to break so.

A

E

A

Like using a capture handle like it's not clear when you use it, you just read something and then you might find it completely useless and ignore it, but the the user agent does not know that you've not found it useful. Once you've read it.

E

Fair enough, I mean again uh like if the hint gets us where we want- and I think it's probably fine but yeah in terms of use cases.

E

Okay, it feels we are adding complexity to a broader surface and the one we really want to address. So that's my only very mild concern. I guess.

A

I mean so once it's a hint, it can also be dropped relatively easily in the future. If we find that it is problematic, I cannot imagine how it would be problematic, but it is possible.

D

So I would flip it and say that um being able to control next previous slide and that kind of stuff, that's the property of the captured page. So I'm assuming, uh if I navigate from one page with this ability to a second one, with the also with the ability that would work really well now with capture handle right. So I don't see a difference between tab, navigation and switching from one presentation to another. So when it works, it sounds like it's great that it you know.

D

Obviously it won't work on pages that don't support it, but I think that's a question of time really well.

A

It's a bit of a question here because let's say that I capture dopes and now I can scroll up and down the dock from inside of me and suddenly I think, like hey, that's a general ability, that's unrelated to like me as a user. I don't really know why it works right uh and then I share this tab instead wikipedia and it stops working now.

A

I think there's a bug now it if I were to capture it from the beginning, you're right that I could still think it's about, because it would never have worked, and maybe it worked before or before navigation.

A

uh It is mostly a concern for self-capture and you're right. There.

F

I would think the wikipedia argument is not very convincing, because the capturing page is uh responsible to show ui uh explaining what is happening like oh there's, no, more controls, because of that, and you can explain it.

F

So um I would think that if a website is uh so, the only case I would see have it happening is, uh if you change the source, the web application discovers it and decides to to stop the track and say no, please uh please go back to to this these kind of things and they can do it, uh no matter the amount of the button so.

A

Yeah I acknowledge again. This is mostly interesting for self-capture, but I think that it is enough if we've got even only two applications that engage in self-capture. One of them wants it, the other one doesn't. If we cannot have a heuristic that keeps both of them happy, then this hint is going to make them happy.

F

Yeah, I think universe argument would be to say that one application would call get display media and the other one. We would call get report media.

A

I would be I'm gonna be very convinced by this argument. Once again, viewport media is implemented and uh in all browsers and adopted by web developers, because at the moment- and this has been reiterated for the last year and a half while we've been talking about- get viewport media, we don't even know of applications that can actually um that so get hyper. Media has two requirements. One of them is not even finalized. Mozilla is actually everybody in was look.

A

Some people in mozilla other than univr have not actually consented to that mechanism being introduced, let alone been uh the get viewport media gay team. So there are a lot of ifs here and I would be surprised if get viewport media comes in the next in less than a year. uh You never. You could contradict me here.

D

Well, I mean I would also love to see chrome implementing this and uh chrome has no such uh blocker on mozilla and also, I don't believe it's true that uh I think we are amenable to uh document policy, at least in this case. So I don't try to say that we're blocking on that.

A

Okay, and do we have uh we've got a lot of web developers that say that they would not be able to adopt this, so we will have to keep the display media for a while, and even if we moved with all possible speed, don't get viewport media which of course we should. It would probably still take quite a long time yet.

D

So my main concern is that we keep adding features to get display media for something we know is unsafe, and that seems in some respects, if I were, uh I'm worried that we're getting farther and farther away from implementing get viewport media.

A

I understand I've. I've got the inverse concern. I've got the concern that for a year and a half now we've been discussing get viewport media and it was used as uh rationale for not doing certain things and even though you know a year and a half later, and it's still not here, and we don't know if it will ever be implemented or will ever be adopted, and we cannot keep on discussing this.

A

I think that whoever wants to champion get viewport media should go or steam ahead and once they've done that everybody else are gonna, get get on board.

A

But we should use the carrot of here's get viewport media, not the stick of hey. You cannot implement other things because of getting because of the vision of get viewport media.

D

I don't think there are any big outstanding issues that would prevent any vendor from implementing it. So hopefully we could see progress there and I agree. That's that'll be good um as far as um it sounds like if this would be a hint that we could uh uh deprecate eventually, uh that might be okay.

D

I would like to see a path here toward assuming get viewport media happens that I would hope that browsers would vendors would try to uh make this move, and uh maybe this could be deprecated at that time.

A

Is anybody uh taking minutes.

E

A

Okay, so it seems like there is consensus on making this a hint. Am I right.

D

Rough agreement, I guess well, I I'm not. I'm not super excited about it, but I I don't see a need to to block it either.

A

D

I'm also done it doesn't put any onus on other vendors to actually implement.

A

It correct that is correct, even norton, chrome itself who suggested it. We might also not implemented it's unlikely. We will probably, but yes, nobody has to implement hints.

A

um I don't think there's anybody after me in the cube bernard, do you have any wrap up items or.

B

It's only 12 minutes left, so I'm not sure that we can do uh capture much more in this meeting.

B

So if we're done, I think.

B

We probably wouldn't call it a a day.

A

That would work for me. uh I guess that if we've got more time, we can just uh maybe if you never could suggest what kind of shape he would like. Maybe if everybody else is happy with it, we could just record that we've made a decision.

B

uh Didn't we skip some issues, we did um but they're kind of complicated, so I doubt we can get through them in 12 minutes all right.

B

If people really want to talk about that, yeah.

G

Just just to be clear, like I filed a lot of those- and I am not.

B

G

To be able to make the next meeting.

B

Okay, uh so you wanna, you wanna talk about some of those byron.

G

Yeah, I mean very briefly: I I realized that, like hammering out, any kind of solution is probably going to take way more time than we have.

B

Why don't we just try to introduce it then uh try to put you know, I think they're all related. As you said, yeah.

G

They're, all tangled together.

B

And I don't know if you like the slides, but um where do you want to start? Is there one that you think might be.

G

um So I think the previous slide that you were just on is is fine um and I'm not really gonna follow the slides one by one. Because again, I think we are short on time, but ultimately, what all this cluster of issues boils down to is um the simulcast ietf spec basically means that we have to allow a remote description to remove a red or remove a simulcast encoding at will.

G

um There's nothing in that specification that allows us to say you know what we're not going to allow a remote endpoint to either reject a simulcast encoding in an answer to our offer or to forbid a remote endpoint from issuing a new offer, a re-offer that removes a previously negotiated simulcast encoding.

G

So uh I think that some of the language in webrtc pc was written with the expectation that these things not be allowed, but there's also parts that seem to allow it. um Although some of that language is aimed at an initial answer, not a not a re-answer, so.

G

Basically, what it boils down to is: uh we've got to decide how we're going to handle handle this uh discrepancy, um and I kind of outlaid or laid out a few different um things that we need to keep in mind. One is, of course, that we just have to allow the removal of simulcast encodings, initiated by a remote description. There's just not really any way around that. I don't think.

G

As far as adding new simulcast encodings, due to a remote description in a renegotiation, I think that there's not really any reason to allow that, because the spec doesn't force us to allow it. You know if an if a re-offer has a new encoding we're under no obligation to accept it, and so I think that we probably should reject it um and, of course you can't add a new encoding in an answer that wasn't in the offer.

G

That's just that's just a syntax error, um so uh I don't think unless there's like some concrete use case for allowing the addition of things to the simulcast envelope after the initial negotiation, I don't think that we should allow it.

G

I don't know if anybody has a concrete use case for that.

G

Does anybody think that somebody has a concrete use case for that.

B

I couldn't think of one.

G

Yeah, I mean I'd, say unless somebody comes up with something pretty compelling, uh I think it makes sense to just disallow grids to be added in renegotiation. um The other wrinkle that's a little bit harder.

G

G

Sdp negotiation can reorder rids at will in a simulcast attribute.

G

G

The the the meaning that the order has in a simulcast attribute is related to transmission priority, um which we aren't really paying attention to right now in webrtc, pc we've got an extension spec that allows the transmission priority to be uh deliberately specified from uh from the javascript api and the language in the simulcast spec around uh how you interpret the order of reds in the simulcast attribute is kind of hand, wavy and there's a should level.

G

uh There's some should level language on how to interpret that, but because it should level we could probably get away with saying no we're not going to do it that way, we're going to explicitly set it, um which is fine, but if we do get a re-offer with rids in a different order than they showed up or were negotiated previously.

G

I think that we kind of have to use the same order in the answer. Even if.

G

That doesn't end up impacting the order in the send, including encoding slot, uh which is something to keep in mind. I guess- um and I think I mentioned that trying to figure out which issue that I kind of lay all those out and it's one of these yeah. So it's two seven two three uh I could lay out some of this comment.

F

Would it be possible to simplify this and just say that we'll reject if the order is different or is there any use case for it as well.

G

So I don't so what you're proposing here is throwing like an invalid modification error or something is, is that what you're proposing yeah.

D

G

About was that a yes.

F

G

So um that might be acceptable. um I doubt that you're gonna see any simulcast receiver implementations just kind of fiddling with the order of the reds. Probably um so maybe it wouldn't cause too much trouble, but it is syntactically, valid, stp and just barfing on. It is probably a base, spec violation, although you could just say you know what we're we're not even going to try dealing with this.

G

um I think that.

G

Just specifying that you mirror the order whatever that order happens to be and don't mess with sending codings at all is a sensible thing to do and not terribly hard to specify. Probably, um but that's something to keep in mind um so.

G

That's really the essence of all of these bugs um is issues around modifying the simulcast envelope and rid renegotiation.

G

And then I guess there's this other one where we have. uh This is 2722 where a re-offer can stop the sending codings entirely, which is yeah.

B

That just was a mistake. I think yeah.

G

Yeah, that's just a you know, that's a and I think janavar has already sort of proposed some language somewhere. um So that's probably a different issue, although it's related, um so that's pretty much every everything I have to say on those.

B

So what is the next step here? uh Maybe maybe we can ref in the issues we can refine the recommendation.

G

Yeah, I I think I think, continuing the discussion in in the the bugs, particularly uh in bug 2723.

G

uh If people could, you know kind of pile in there and and read that comment and give your take on what makes sense to do here. uh We can start hammering out a pull request.

G

And that's, that's me done. um Okay,.

B

um Well, thank you uh and I would encourage people. uh I guess uh to take a look at this. It is a complicated set of issues, but um I think we've become uh aware that the samakast aspects of the spec do have some problems, as we discussed.

B

Okay, well, thank you.

B

All right thanks, okay, so I think we're done for today. um As we've said, the next meeting will be on june, 7th and we'll be taking agenda.

D

B

So if you'd like some time, let us know thanks: everybody.

E

Thank you, bye-bye bye.

E