Nebula Graph NebulaGraph for Beginners, 20 Jul 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 新手玩转 Nebula - Data Import Options (CN Version & EN Subtitle)

Description

在本视频中你将了解到 Nebula Graph 的多种数据导入方式，从你的数据源开始，一步步教你如何选择导入工具。同样的，本视频中的数据导入图片你可以通过 Nebula Graph 微信公众号发送「数据导入」关键词来获得。如果你在数据导入过程中遇到任何使用问题，欢迎来论坛：https://discuss.nebula-graph.com.cn 和我们交流哟～

A

Hey ~ hello, everyone,, I'm hera. This is thought to.

A

Hi hello, everyone,, I think, is very glad to meet you.

B

And I am happy and to share content on nebula graph together.

A

Also, please a lot of attention to our video no. yo.

A

ah ~ then come to the question, on this issue,. We finished the deployment of us say that data into it.

A

Thinking for the first call to tell us about the data, import methods and tools, nebula graph, currently provide.

A

B

Today, we introduce how importing data nebula graph mode selection offers a wide variety of.

B

Tools and libraries to help us to import data from different data sources to graph the cluster. Where.

B

We choose and how to distinguish them?

B

first,. We simply look at where we have three.

B

Options,, the first is nebula importer. nebula importer is golang, write a.

B

Single executable file,, which can help us run the csv file into in accordance with established rules on a single machine.

B

The nebula graph in followed nebula exchange it is applied on a spark, allow us to not write code or a small amount of write code to call the library.

B

Batches to import quantities of data from different sources into the nebula graph.

B

Last is nebula flink connector. It is a software package on a flink flink help us.

B

On the platform,, the data stream read and write operations and then imported into nebula graph.

B

So how to distinguish them?, we can answer these questions from first, start.

B

The first question is whether our data source is a file?.

B

If our answer is: yes,, we can do this. The lower branch of.

B

The second question is whether it is a csv file?. If that is the case,, we have to do if we import.

B

A branch of the running workload can be used stand-alone to meet our memory on the requirements?.

B

If it's not a big amount that we do not need more than a single machine,, then we can use the csv file. Nebula importer.

B

Would be sufficient to meet our needs, and that if we have to call to more than one stage of computing power.

B

And connectivity to the import data from a csv file to nebula graph.

B

Nebula exchange, then we should use it if it is not a csv file. Nebula exchange.

B

There are just a few of the other types of data source file.

B

Back to the question of our files,: if not,, then we can answer.

B

File, the question is whether we need a number of warehouse or database to import it?. If the answer is: yes,, we are use to exchange.

B

That exchange is currently supported, neo4j, if it is introduced into the data stream of the four.

B

Cases in four different database server and several warehouses in.

B

Mysql, hive, and it also hbase two cases,. If we are from pop / sub.

B

This message, bus platforms, such as kafka, pulsar or.

B

To import it,, we use exchange it. If we are to import data stream from flink platform,, then.

B

We should use natural flink, a connector.

B

So these simple conclusion is that we can make the selection by answering these questions.

A

ah ~ okay,. Thank you very much for the detailed description of thinking.

A

ah ~, I can understand it? in the case of streaming data if the data source is flink.

A

On with nebula flink connector, if kafka or pulsar this message, bus over the data.

A

On the use of nebula exhcnage right? good entirely.

A

Correct, that there is a problem, you import tool described above.

A

And we nebula client write what difference? Does it make?.

A

And which one is more recommended?

A

ah ~ in fact,, they are used in different scenarios,. That all depends on. The application form is with the communication of its.

B

Nebula graph background of your application.

B

For example,, if your application is its source of information, is coming through a rest. Api.

B

That natural you do this, you should insert such a.

B

Corresponding write this with you, the language of the web server.

B

Client., because it is not a particularly large amount of data,, but if your sources are, for example,.

B

Mentioned before ah csv or massage bus, ah,, ah, or flink natural, that you should mention that we have just a few.

B

That you, natural should mention that we have just introduced several tools to do it.. That is, if your application.

B

Is running in the spark itself or graphx or run flink inside,, then you should.

B

Call us the appropriate connector platform computing platform directly inside your application, such as.

B

Spark spark connector is, then flink is flink connector. So when you know all these things.

B

What they actually mean when you will naturally choose the appropriate way to go ah ~ okay, thank you.. Then there is the next problem.

A

Is the difference between kazakhstan and exchange spark connector of what is it?.

A

ah ~, this is a good question.

B

Right, I did not mention before we nebula another spark connector I have not listed here. It is somewhat similar to.

B

It, except that it is somewhat similar, flink connector, is on a spark platform,. Then it. What is the difference with the exchange we already have?.

B

The difference between them is spark connector. Does it actually.

B

Like flink, it is actually a package api library on a spark,, and if there is a demand, we.

B

Need to read operations to write with map data, from nebula graph inside on the spark we should.

B

Otherwise,, if we just make this an imported one working with spark connector the library,, then we will use the exchange may be that.

B

We can be submitted directly to the spark job without writing code.

B

Or can call this, this, in the spark of the client which exchange this job.

B

So probably can understand the difference between the two?.

B

I have one last question.: what are the ways to write to exchange?.

A

ah ~, this is a good question.

B

Next,, the problem is also very good.

B

If I introduce the exchange of words. internal exchange, it,, there are actually three different component are reader.

B

Are actually three different componet? They reader.

B

Processor and wrtiter, that literally means we know the purpose is to exchange.

B

The incoming data into the nebula graph in it,. We also know that the exchange supports a variety of data sources.

B

It was first read in the processing and then import such a process, that reader, if.

B

Our different reader of this class to help us deal with different sources. first of all,, the first one is serverbase.

B

That literally means we are dealing with this type of server is the source of that fact,.

B

These positions, corresponding to the number of servers or database file base, is similar to that used to process files based on a source file. Such.

B

That streaming base stream data that is processed after kafka and pulsa.

B

That reader to read it yet,, go to the processor inside.

B

ah ~ there is, there is a little difference,. It is that it depends on what way we would like to eventually go to write, nebula graph.

B

It depends on what way we eventually written into the.

B

Nebula graph nebula graph, we know, is a distributed map database,. Its underlying storage is based on.

B

Our rocksdb that rocksdb, we all know,. If we have to write ultra-high demand,, then.

B

We can support structure, sst files directly injected into rocksdb inside,, so to avoid those of the session in the middle of.

B

Kv level of the line level that it is very, very fast.

B

So the answer to this question: is, ah ~, what is written way?? There are two.

B

One server base, a file base server base it is, in fact, deal with it in the process becomes a data insertion of vertices and edges in the manner points and edges. After the fact.

B

By writer, server bace nebula graph of graph client is then equivalent to the situation as it will optimize. Our treatment to clients in the batch is inserted into the map database.

B

That is if we have a really huge amount of data, and then we use the server base has been unable to meet the requirements of.

B

We can also use this case based on the file base of the writer sst, in this case,.

B

It was the middle of its process, is sst processor. Then it is written to the.

B

Nebula graph years,, so there is a good.

B

Way to write two very xie xiesi explanation for today.

A

I hope you understand that, after this information can help, you select the import mode data.

A

If relevant, doubts or questions,, you can leave a message. We will get back to you next issue, then goodbye.

A

In the video below see the next issue. see you in the next issue~.