Magento Multi Source Inventory, 8 Mar 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Magento MSI Open Demo. March 8, 2019

Description

Agenda:

- Elasticsearch 6 support for Magento 2.3.1 and 2.2.8
- Modularity discussion
- MFTF for MSI

A

Yes, recording his talent, so hello, everybody and welcome to magenta missed our weekly demo meeting today. We have kind of diversity topics.

A

Today we all talk a little bit about elasticsearch six, because we like me and the especially huge influx of our work involved into this topic during this week and one of one of the msi contributors who is currently connecting to our medium made this contribution so that that's why I thought that I'm a psychic, the demo, could be a great place to make this demonstration and share with the update and because also we tested nulastic 36, whether it's support or not msi, especially taking into account that and msi 1.1 dot.

A

All we introduce full support of elastic search, elastic search engine so that we also verified that messiahs fully supported with elastic search six but IC determine already joined and yeah.

A

So, let's not introduce any changes to the initial agenda, which is shirt in the messiah channel and we can start with remains presentation and organ remains presentation on his like great contribution and because he was actually it's like a fairy tale story, because we were able to deliver at that at the latest week of preparation for to the 3.1 magenta listened to the 2.8 magenta released, we were able to fill the gap and introduced. The support for elastic 36 tends to remain and remain.

A

If you hear me well and you can, you can draw it'll start sharing your screen, and this stage is yours.

B

Hello, do you have to okay great I had some issues to join the meeting since I'm, usually on Linux I have switched to a computer on Windows, so I it will be a bit complicated.

B

But I do my best.

A

So because we have product on our own a call we have Anton Krug, who is also is not so good reading the code so but a.

B

Request to Magento, because we found that elasticsearch Friday was scheduled to be at the fly fund, March 11th, so it's Monday and I also also people and github, and on the community engineering slack, which were wondering if Magento will support the new version. 6 of elasticsearch, so I found it pretty cool to this topic. Since.

B

Implementation.

B

B

Did some help before with the elasticsearch 5 implementation last year.

B

What this reward? Maybe it's things with.

C

B

Which is called underscore all which was existing on previous version of elasticsearch despite being deprecated, but since version six, it has been completely forbidden to use this as a catch-all for history.

B

So for now, in the implementation of six, the general part which has resolver, which is my big condition written if the search request, dot XML file of written by the user of a search criteria, its mapping fit name to fit name on elastic search. So there was existing mappings between staff characters, which was previously mapping to the old field.

B

So this part was to cool down by replacing on the flight request on the old field by new field, which is called and Oscar search, which has the.

C

B

Old field, which is a catch, so this is for the deepest part, but.

B

B

Big part, which is this one, and so for elastic, search six when I declared a ping I have no a field which is called search, underscore search, which is used the same way as the old Rosco old field. Word was used, it's a catch-all field and you see that we have on the dynamic templates copy to underscore search for the string. Mappings.

B

The rest of the file is mostly the same as when what it was done on the elasticsearch file, my big responsible classes, but since I created a new module I had to copy past. Everything like in the index exist, update, aleeah's, exist, Aaliyah's and so on to mimic the same objects that existed before for elastic subjects.

B

So that was the most important part and the other part is the suggest part which was on magenta to the two related to the advanced search module, but has been merged into community version since two point three on this part, this part.

B

C

B

This one, this one has been refactored a lot because elasticsearch deprecated, the usage of the suggest end point and encourage users to build, suggests queries on the search end point before no. This class has had to be worked point on touch query, which is done via the legacy. Query: a PE of the search connection. Oh I, like this feature.

B

In fact, we loop on the field which are used, it's search and it's it's I'd suggest build yeah yeah. We have a method which is fetching the field which should be used in suggest, so it's done like we're wearing with a field provider for all the fields, and then we filter to keep only the text, fields and indexed field and then for each of the field. We built some query to be passed later on the suggest, search, query and to fetch the suggest wizard.

B

So this one I've been quite a big amount of the walk, but the rest of the integration of elasticsearch six was quite easy. In fact, it was mainly.

C

B

Get rid of the OL capture old field, which has been deprecated so far. Now we have a basic. What I would call a basic implementation of elasticsearch six. It does nothing more than what we had previously with elasticsearch five, but it's but it's up to date with with the engine version with indeed latest stable version. But there is also one point that we have to keep in mind that we already discussed with ego and the second ego is elasticsearch.

B

Seven will also be released, so maybe this time we can push forward and to look how we could support also elasticsearch seven, and this implies also other things. It's actually I think there will be three versions of the elasticsearch PHP client, which will be set as required in the composer, a JSON file which.

C

B

Word and a bit a bit cleaner, because if I'm correct and these people at Magento did not rewrite the composer JSON, it should require elasticsearch.

B

22.5 dot X or six dot X, which is a bit a bit confusing for end-users. But it's it's also strongly tied to the fact that there are still many customers of Magento which are using elasticsearch, especially mentioned to cloud customers. So I know it. It will be a huge huge amount of work to tell the these customers to migrate to new version of puff elasticsearch.

A

Yes, this is this is very correct statement and we already from our point. We already started to communication with our product organization, so that I'm really not sure that we would be able to eliminate or to abandon support for elasticsearch, two or elasticsearch five until magenta 2.4. But we what we'll try to do we'll try to duplicate the support of this. This functionality will try to duplicate, support, exist, engines and the wrong is that we will try to duplicate module where these end points actually located.

A

Currently, so, if you don't mind, I will I I will share my screen and I will show part of you request a part of your pull request and part of your book and contribution, which is pretty interesting, and probably it's gonna, be pretty interesting for today. Setting D so follow a meeting, especially for Antonia and Libby.

B

A

Because I had I heard that Lucas had some concerns regarding the majority part. Yes, the. What what's very interesting in Europe contribution that we introduced support of the elastic 36 as an independent module along quiz that we did not introduce any new API in this module so or all the API is still.

A

Still in whether George or advanced, George or elasticsearch module.

B

We just provided implementation of existing interfaces like suggests queries to face. Oh.

D

B

It's quite decoupled actually can.

A

I can I, please open, or maybe maybe probably, the better brunch, because I have an access to to the 3.1 yeah I to the T. That's why I wrote is brunch. So any please stop sharing your screen. Sorry.

A

A

Yep, so we are currently into the 3.1 release branch. So this is this nap shot of our code base, which is going to be released pretty soon, probably the next week. Okay, great and here you can see that we don't have any dependencies like on anything except short, catalog, search and elastic search and the interesting part that we, what we did.

A

We introduced a new module here, but this new model doesn't introduce a new API is none of the classes which a market API PHP dog blog- and this is this- is done slightly different in comparison to what to how we've done the elasticsearch fifth support, which was introduced, the namespace inside the elastic search of elastic search, because we already seen the Roman demonstrated that part of the request has changed in the elastic search, our six, so that we know really need to help like one module like elastic search, which will introduce some common parts, because we see that, like request is began changed and their response is also being changed from one major version to another version of all versions of the elastic search.

A

So we no need to have like one common elastic search, module and two to depend on it in different elastic search version. So the main idea here so the idea which we want to demonstrate and the last and the architectural console is to have just modulus, which represents support of particular elastic search version.

A

So, as remain mentioned, when, in in a few weeks, guys from elastica will release in general availability elastic search, seven, what we'll just do we can just bring another module, and ideally the elastic search module should be dismantled because currently, currently it doesn't in also does not introduce all api's to what it does. It didn't reduce fbi's, and this SP is our extended by different different.

A

Elasticsearch version module, but this is not correct responsibility and probably this well part of the logic should be moved to the advanced search so like like the factory which introduced all the all the adapters which we currently support and which is extended with elastic search tool, I still search 500 searches at this moment. So this is this is decision like taking module arity into account and just one one more like explicit statement that, ideally, we are not gonna have the elastic search module here, so probably in a time it would be dismantled.

A

Also so I believe it would be replicated along the time so when it will relieve the elastic search, seven support. Hopefully that also would be done as an independent module where.

D

A

We already have two modules, as the one is George and another one is advanced search so initially working.

D

With whatever is torch,.

A

A

We have two modules like.

D

Which is effectively should play a role abstract George, that is the engine agnostic right and we it made a lot of sense because we had elastic a solar. We had my skill and solar them, my scale, solar and elasticsearch. Now we have my now, we have my scale and we have two tangents effectively, but you killed my skill at some point made some from that inspector now. Also elastic search, module kind of made sense component pregnant important made sense because it was like elastic implementation.

C

D

But moving the last extension of what we will do, we will effectively manage elastic, search to search but I'm, not sure that is that's completely good from perspective of staying up start from perspective, switch engine powers search in magenta or even if you have elastic so we're saying that we won't search and we won't search api's, but search can be performed by either elastic or some remotes are searches on my god.

D

C

D

About removing elastic search module or manage elastic search, module, 2/3.

D

B

Could be a module in framework Magento framework which could be imagined to framework elastic search implementation of this module with elastic search, five, six, seven on up code Magento, but.

B

A

Just the two statements, the first one is the search module. The main idea: the search module was not to introduce abstraction on the level of adapters, but that was an idea to introduce abstraction and level of features. So actually all the adopts of the research module is agnostic to all the adapters, because.

B

A

Already in to use own declarative, schema for requests and.

D

A

Is agnostic to any adapter.

D

Defined search a pH, and then we have different Moodle or framework component for me, it's not them.

B

D

Not something that much difference between making it a modular framework, and we have elastic search that implements logic, that elastic search, specific adapter, implementation of search right.

A

This is very correct statement what you just did so it's introduced logics for very specific elasticsearch adapter, because we can't actually separate some API, which would be agnostic to different elasticsearch versions, because we see that guys from elastic introduced backwards, incompatible changes for both request and the response API. So we don't have anything in concrete to put into so.

D

What we are saying is that we have searches at thousand of that knows nothing about elastic search and then we have elastic search, powerful search, takes elastic, search, seven, giving different models that actually implements all full implement new, full implementation of working with elastic search. So now, I.

C

Have a question.

D

If, yes, there might be logic, some logic that is completely incompatible, but there it might be, and will probably be some logic that stays the same between vs. yeah.

A

It's mine, it might be, it might.

D

So where do we put that logic, if you kill elastic search model.

A

The question is whether.

A

Yeah so I hope that look is completely against but yeah. So what what's the desirable state of this idea? Is it we? Probably? It would be pretty similar to what we have with HP versions.

A

So at the time when magenta is being released with support, just maintained version of elasticsearch in the market, so that probably could be done from magenta magenta to that four or later versions, and the idea that we can pretty easily to duplicate and eliminate part of the business object from our code base and currently like even here, introducing the elasticsearch six module.

B

A

Not introduce any particular API to that module, so we do not introduce all the extension developers are still supposed to rely on the same set API which were existed before.

A

C

It's like seven years ago, five years ago and adapted to.

C

A

You can't decouple because we are not in control of elasticsearch, so elasticsearch introduced backwards, incompatible changes in different version of elasticsearch. So, for example, the request API of elasticsearch changed from version to version 5 and from 35 to version 6 and from version 6 to 7.

B

A

If you have an.

A

Object, for example,.

A

You can have an API, because the the code under the hood of the API is so what we, what we will be provided. Actually we will be providing the modification of the elastic source because the name the module is rustic Forge. We can't introduce our bloody.

C

Wants to castrate, we.

A

Eventually, not right now, though, it would be because currently what we see, we don't have any particular API in the elastic storage module. We have several factors which are extended like I can show you. We have several factories like this one like different elasticsearch model config, and we extend it with with supported engine list and the elastic shorties introduces support of the elastic.

D

So we in this question, I think it's very important question. It's not related to inventory I think it's related to multi service inventory by zero percent, so I suggest to have.

B

D

Meeting to discuss it Monday with wider audience, it's important question and there are some good arguments on both sides. I suggest now to move on to MSI topics and have set up separate meeting to discuss this question. Okay,.

C

D

Have to jumble the main reason, because I have to jump to the next meeting. I would like to discuss.

A

I've seen that you already pack your backpack with swag and now you're, going to I.

D

A

To jump to different meeting notes out of room but yeah and they say say hi to all other guys.

D

But yes, Lucas! If those are good points, let's discuss some on Monday good one day works for everybody, but anyway, so.

A

Okay, so think in like in any case like this, is I just want to see it once once again that this is kind of a fairy tale story.

A

When we we had a gap, we probably will meet that we have support robotics at six him to the 3.1 and we means that the support of elasticsearch five is running out of time and we like remain track set, and he brought this contribution and we managed to and one more important thing which I wanted, which I want to see that we migrate all our tests to elasticsearch six and currently internally, we run all the tests on the Jenkins environment on the elasticsearch six.

A

So before that, even having support of the elasticsearch five we actually we had. We had an issue because what we did on our Jenkins environment, we were running elastic storage, test on the elastic search too. So even the elastic search five it should be. We did not abridge by the test with the elastic search file, so we were still using our continuous integration environment.

A

We were still verifying that elastic search too is working, but currently we change the machines, both the environment, which is which change both our integration test and now we're verifying that everything works as expected on the supported and maintained version of the elastic search, which is very great. So this is, as Anton mentioned. We like we spend quite a lot of time, but because it's very very important topic to discuss so now, let's switch to another great topic, don't remember that. Let me check the agenda.

A

Here we have yeah yeah here we here we have Lucas so Lucas. You already told that one one thing you are strongly strongly against. We so tell us another thing: you are strongly supported because you are something.

C

That it would be better if you.

C

A

I'm not sure that I can run any MFT f-test right now or should I go to.

A

Well, yeah: you can.

C

C

C

We can remove it.

A

Remember our conversation, probably the day before yesterday, where you said that currently there is no the latest main line in the magenta main line in the EM assignment line.

A

So what we did we we are going to merge this pull request today and we're going to introduce the lathe to the tree to a messiah so that you can benefit of using all the features introduced to the HTF and you can easily contribute to the MST of tests on msi tracking, particularly so we just like we're filling the gap, we're listening yeah and this is going to be so. You can created this request so that we will merge it today and the msi man.

A

One will contain the latest to the three develop branch, including the MFT f-test, all the only new part of the emir TF test racing.

C

A

Thank You, Lucas and I hope that you would be the same enthusiastic as you are currently even after the match test fest, and you will continue working on improving yourself and improve in magenta, though improving us with your contributions.

C

Yeah right now to catalog search module. That's why.

C

I spent five months on elastic search and we customized that work with custom. It's like range and that's why I'm so not having possibility to extend elastic search module anyway, except doing forensics so I have some concepts to discuss on Monday.

C

A

Like just just to set you some lights that we are going to spend more time on, even besides the fact that will discuss how to support properly elastic search, several viewers- and we want to discuss with the future of the of the search in magenta and how to support properly and how to be how to make our search a better place, how to make our search, much more precise, more extendable and so on so forth.

A

So we will be scheduling, like I, believe, a series of meetings groomings, where we will be discussing how to help you introduce new functionality to search and, if I'm, not wrong. You're going to be on this magenta you magenta meetup in Krakow, on 28th of March and on the authority of March as well. Is it true we I'm coming to that event, and we can even work shoulder to shoulder elbow to elbow on a search contribution. I.

C

Will be 28.

C

B

A

C

A

You're not joining to the event which is organized by its tricks.

A

By your employer, but you turn in another event, organized by by other guy from.

C

Contributes- and it just talks me like two weeks before.

A

Okay, I got your point, but in any case, if you, if you can join remotely so that you, because I thought that we will be able to discuss that personally and I think how we can improve the search but yeah. In any case, we still have a plenty of time to catch up.

C

A

The that was another, so let's get back to the agenda, because I am not really I'm, not really following the agenda right now, so the general updates by me. So what what? What I wanted to share with you so that you can introduce that fix to to try this. So this is supposed to introduce like this supposed to fix our integration test on Travis environment, because he, as you know, we still have some issues with with Ronnie integration tests on some engines.

A

That's why our our integration tests are run out of time before and they they not have enough time to proceed and that's why their engine on 98% of execution and that's gonna be fixed pretty shortly by UVM, shock, saguaro and also pretty interesting updates, so update from ricardo Tempesta.

A

So we are still working on this pull request and he just pinned me today that he spent a half a day during the holiday during the match test first contribution day and looks like he fixed all the issues we found and like just just to introduce you a little bit more background.

A

Currently, we've seen that in MSI we introduced performance degradation of the products. Evaporation, and this has happened because we at the time of the products beyond save. We have to synchronize the inventory data to all the other storages to the new one which is being used by MSI and to the legacy one.

A

We only used by the catalytic inventory so that we have this plugin ization and we are launching the new api, we're launching the previous api and so that, instead of saving like let's say, inventory, one time we do it twice like we are the new EP and we are the all the periods of this introduce a performance degradation because we are doing the same work twice.

A

So what we decided to do because the product save is pretty crucial part- and this is done by like this damn on a day-to-day basis as it's done during the import/export operation, so that we definitely don't want the situation to be to be slow. So what he decided to do. We decided to put all that synchronisation apart into the into the and to make it as in corners.

A

We are there a bit in queue and the initial request of Ricardo, which he which he mentioned about on a previous demo, so he already had some prototype, but we tested this prototype during this week. Also we introduce Alex resume and I'm not sure whether he is currently on on a meeting, because he just returned from his long trip to Norway where he was like hiking and as soon as he got back from Norway I asked him: hey man.

A

Can you can you please take a look at the pool request of Ricarda because you're pretty good at there about all this I think stuff we have in magenta, so Alex also had some some ideas: how to improve the sport. Well, so we sync up and today a record introduce the changes. You can see that the builds are still running on his pool request and he yeah, and he asked our politicians engineer, Slava to check it out whether it's still whether it's still broken or it's it's getting better now.

A

So we will definitely we'll definitely keep you updated on this blue plate, because this is pretty crucial part and this part effects too much work. That's why we are. We are reviewing it and testing it really carefully, because potentially it in it could introduce too many too many potential issues. That's why I will be keep you updated and probably on the next MSI human meter will also show you update regarding organize a stop another.

B

A

General update I'm not sure whether step on step on currently is on the cold topo I here, if not I'll I, will make an update on behalf of you that looks like Stephane is not currently on a call. So on a previous on a previous meeting on a previous MSI, a weekly meeting I asked pre you from ask Priya from the tales channel project to join our medium and made an appeal for MSI contributors to ask: introduce this inventory stock expert functionality. So we didn't have the functionality before so that what we have.

A

We have the source, important expert, but here we want to export the aggregated data and including and taking the reservation into account, because this is exactly how we gonna integrate with Amazon, with Google shopping, with Facebook marketplace, with Walmart and so on and so forth with all other marketplaces.

A

So we want to. We want to consider like magenta to play the role as an ERP system, build the build our gated stop data and export the stock data to this particular sales channel, and this deponent reviews the pull request where he created the initial draft of this functionality where he introduced the API, and currently we are working on stabilizing that API and, like I, hope that you're enduring the next meeting. So here you can see this API, so I hope that you know the next meeting.

A

Will we already make a demonstration of the working prototype of this expert? Aggregated stock expert functionality is working and I'm just looking whether I have another.

B

A

Updates to share with you guys don't.

B

Do them during.

A

The boom yeah and we also had two issues which were which were raised by when I cop in the EMA satisfaction or regardless, that we have some some unprecedented.

A

Some accounting and, for example, it is a pretty interesting case when, when I created create the product with the call quantity of two also he allowed the book order for the product he he applies. The use decimal quantities on and also he applied, the out of stocks or Satan s, minus 1. So the according to the MSI logic we have the saleable quantity of this product is 2 minus minus 1 which give us 3 altogether, and then he plays nine orders and all of these orders were decimal quantities.

A

So if you just make it total, if you just sum up all these values, the total supposed to be exact 3, but currently because we we had a gap with visa calculation and there with a custom of quantities to decimal one. The last the last order for a fraction of 17 so couldn't be placed and because of the wrong conversion. So the Lars introduces fix and.

A

Yeah and we are almost ready to accept that- to request.

A

So I believe this public was gonna, be merged pretty pretty soon, and this is the short of short general updates on all the pull request. Yet.

B

A

Also hit yeah well so head pretty interesting idea from yeah, but but it was not created as a separate pool request, yeah, so probably more news to come on on the next meeting. So this is pretty much all from our side regarding all that happened in MSI and the stuff which is related to MSI. If there are any any open equation, just don't hesitate to ask us. Maybe if you have to question right now, just just ask it right now: if no you can always, you can always drop that question to the MSI's like channel.

A

If you are not in this lecture, feel free to drive such MSI slack channel and probably I'll, actually pretty interesting, update, I'm trying to make guys from Lima to come to the next demo to come to the next MSI, give a meeting and the share with their update, because I know that they they a lot luncheon and a side based website on March 12.

A

So it's gonna be on Monday and I really hope that they come to MSI weekly meeting on Friday and share with what they did have the customized time and science share with their feedback, whether it was whether it was easy or there we are struggling with a machine of the logic we introduced. So I hope that we'll have the interesting presentation on the next week. But now it looks like we we run out of time and we are running out of all the topics we wanted to highlight with you.

A

So thank you very much for your attendance. Thank you very much for your time and EU see you in a week.

C