如何在nodejs中正确保存scraping到Postgresql数据库的信息

a14dhokn  于 2023-03-17  发布在  Node.js
关注(0)|答案(1)|浏览(136)

我是nodejs的新手,我有一个scraper,我想把数组保存到postgresql数据库中,问题是在我当前的代码中,追加数组比scraping快,这意味着我追加了一个空数组到数据库中,我得到了数据库中的空字段。
我怎样才能使它发生,刮必须先完成,然后追加?

const cheerio = require("cheerio");
const axios = require("axios");
const match_data = []

const url = "https://www.covers.com/sports/nba/matchups?selectedDate=2023-03-08"

const { Pool } = require('pg');
const pool = new Pool({
  user: ###,
  host: ###,
  database: ###,
  password: ###,
  port: ###
});

pool.query(`
CREATE TABLE IF NOT EXISTS nba_test5(
    id varchar PRIMARY KEY NOT NULL,
    hometeam varchar,
    awayteam varchar,
    homescore varchar,
    awayscore varchar
    )
`);

async function getMatch() {
    try {
        const response = await axios.get(url);
        const $ = cheerio.load(response.data)

        const match_list = $('.cmg_matchup_game_box.cmg_game_data');
        match_list.each(function(){
            match_id = $(this).attr('data-event-id');
            home_team = $(this).attr('data-home-team-fullname-search');
            away_team = $(this).attr('data-away-team-fullname-search');
            home_score = $(this).attr('data-home-score');
            away_score = $(this).attr('data-away-score');

            match_data.push({match_id, home_team, away_team, home_score, away_score})
        });
         

        console.log(match_data);
    }
    
    catch(error) {
        console.error(error);
    }
}

getMatch();

pool.query('INSERT INTO nba_test5 (id, hometeam, awayteam, homescore, awayscore) VALUES ($1, $2, $3, $4, $5)', [match_data.map(data => data.match_id), match_data.map(data => data.home_team),match_data.map(data => data.away_team), match_data.map(data => data.home_score),match_data.map(data => data.away_score)], (error, results) => {
    if (error) {
      console.error(error);
    } else {
      console.log('Data inserted successfully');
    }
  });
unhi4e5o

unhi4e5o1#

您需要awaitasync函数调用。如果您不这样做,它将不会在运行插入查询之前等待擦除完成。有关详细信息,请参阅MDN await article
getMatch() -〉await getMatch()

相关问题