How to process 70w+ data.

  mongodb, question

The data structure is as follows: 4 levels at most. There are 70w+ data. Now I want to query and classify them. Classify by city area. My idea is to find out at one time and deal with it recursively. But now there are problems. I use itmongoose findWill time out; How do I deal with these data to achieve what I want?

{
 "_id" : ObjectId("59df0ea9c43b39bc29e7355e"),
 "id" : 1,
 "parent_id" : 0,
 "level" : 0,
 "area_code" : NumberLong(110000000000),
 "zip_code" : 0,
 "city_code" : "",
 "name": "Beijing City",
 "short_name": "Beijing",
 "merger_name": "Beijing",
 "pinyin" : "BeiJing",
 "lng" : 116.407526,
 "lat" : 39.90403
 }
 {
 "_id" : ObjectId("59df0ea9c43b39bc29e7355f"),
 "id" : 2,
 "parent_id" : 1,
 "level" : 1,
 "area_code" : NumberLong(110100000000),
 "zip_code" : 100000,
 "city_code" : 10,
 "name": "Territory",
 "short_name": "Beijing",
 "merger_name": "Beijing",
 "pinyin" : "BeiJing",
 "lng" : 116.407526,
 "lat" : 39.90403
 }
 {
 "_id" : ObjectId("59df0ea9c43b39bc29e73560"),
 "id" : 3,
 "parent_id" : 2,
 "level" : 2,
 "area_code" : NumberLong(110101000000),
 "zip_code" : 100000,
 "city_code" : 10,
 "name": "Dongcheng District",
 "short_name": "Dongcheng",
 "merger_name": "Beijing, Dongcheng",
 "pinyin" : "DongCheng",
 "lng" : 116.416357,
 "lat" : 39.928353
 }
 {
 "_id" : ObjectId("59df0ea9c43b39bc29e73561"),
 "id" : 4,
 "parent_id" : 3,
 "level" : 3,
 "area_code" : NumberLong(110101001000),
 "zip_code" : 100006,
 "city_code" : 10,
 "name": "Donghuamen Street Office",
 "short_name": "Donghuamen",
 "merger_name": "Beijing, Dongcheng, Donghuamen",
 "pinyin" : "DongHuaMen",
 "lng" : 116.406708,
 "lat" : 39.914219
 }
 {
 "_id" : ObjectId("59df0ea9c43b39bc29e73562"),
 "id" : 5,
 "parent_id" : 4,
 "level" : 4,
 "area_code" : NumberLong(110101001001),
 "zip_code" : 100010,
 "city_code" : 10,
 "name": "Dorothy Lane Community Neighborhood Committee",
 "short_name": "Duofu Lane",
 "merger_name": "Beijing, Dongcheng, Donghuamen, Dofu Alley",
 "pinyin" : "DuoFuXiang",
 "lng" : 116.412747,
 "lat" : 39.923014
 }

No matter which database has the same attitude towards data operation, only take the parts you need as much as possible. Apart from making reports or exporting data, it is rarely true that so much data is needed, so the first question is whether you really need 70w data (because I didn’t say what I was doing, I just speculated from experience. Please explain, if possible, what are the specific functions of these data?
For overtime, I don’t know what the conditions for find are, how much data there is, and whether the index is appropriate, so it’s hard to say what the problem is, either in the database or your application can’t handle it.
Finally, talk about the data itself. Judging from the data, what you have to do is probably put these data together into a tree diagram. But I don’t know how you plan to use it after you spell it. such as

  1. KnowBeijing, DongchengFind out all the streets below (from top to bottom);
  2. KnowDorothy laneFind out streets, districts and cities (from bottom to top);

There is nothing to say, according toDorothy laneThe first few levels can be found directly. The more troublesome is 1. But the designer of the database has already thought it out for you. You can communicate first. Or the example of 1 above:

DB. < Collection >. Find ({MERGER _ NAME:/Beijing, Dongcheng/})

Can be found outDonghuamenAndDorothy lane. This query can hit the index

db.<collection>.createIndex({merger_name: 1})

If the above guess is wrong, please explain the more specific requirements.