csv Buffered writer stopping at 4000 [关闭]

ff29svar 于 2023-10-13 发布在其他

关注(0)|答案(1)|浏览(92)

**已关闭。**此问题需要debugging details。它目前不接受回答。

编辑问题以包括desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem。这将帮助其他人回答这个问题。
8天前关闭
Improve this question
我试图将大量数据写入CSV文件，但我使用的BufferedWriter似乎只写入到第4000行，（大约有1，500，000个条目）我有一个散列表，它以带有getter函数的Patient类的形式包含所有数据，而patient只是按患者年龄排序的散列表的键值，但由于某种原因，它没有全部写入
我已经测试了writer之外的所有内容，数据都在里面，条目的数量始终一致。我甚至以采取的步骤数量和需要的数量的形式打印出进度，它显示了每一步。
在网上看过之后，大多数人认为这样的问题是由于没有冲洗作家，但我已经尝试过在写作之前和之后冲洗以及两者，这似乎并不能解决这个问题。
下面是函数：

private static void writeSortedFile(String outputPath, int[] arr) {
        DecimalFormat dec = new DecimalFormat("#%");

        try {
            FileWriter fileWriter = new FileWriter(outputPath);
            BufferedWriter bufferedWriter = new BufferedWriter(fileWriter);

            // Write header line to the CSV file
            String header = "";
            header = "VAERS_ID, AGE_YRS, SEX, RPT_Date, DIED, DATEDIED, SYMPTOM_TEXT, VAX_LIST, SYMPTOM_LIST";
            bufferedWriter.write(header);
            bufferedWriter.newLine();

            for (int i = 0; i < arr.length; i++) {
                if (i % 10000 == 0) {
                    System.out.println("Writing step " + i + " out of: " + arr.length + "\t"
                            + dec.format(((double) i / (double) arr.length)));
                }
                if (map.get(arr[i]) != null) {
                    Patient v = map.get(arr[i]);
                    String age = String.valueOf(v.getAGE_YRS());
                    String ID = String.valueOf(arr[i]);
                    String vaxList = " ";
                    String symptomList = " ";

                    for (Vaccine vax : v.getVAX_LIST()) {
                        vaxList += vax.toString();
                    }
                    for (Symptom symptom : v.getSYMPTOM_LIST()) {
                        symptomList += symptom.toString();
                    }

                    String[] data = { ID, age, v.getSEX(), v.getRPT_DATE(), v.getDIED(), v.getDATEDIED(),
                            v.getSYMPTOM_TEXT(), vaxList, symptomList };
                    String dataLine = String.join(",", data);
                    try {
                        bufferedWriter.write(dataLine);
                        bufferedWriter.newLine();
                    } catch (IOException e) {
                        e.printStackTrace();
                    } finally {
                        bufferedWriter.flush();
                    }
                    
                }
            }

            System.out.println(outputPath + " Written \n");
            bufferedWriter.close();

        } catch (IOException e) {
            e.printStackTrace();
            System.out.println("CSV Writing Error");
        }
    }

下面是函数的输出

TEMPID ARRAY SIZE: 1579584
Quick Sorting by Age....
1575584
Writing Sorted File.... 
Writing step 0 out of: 1579584  0%
Writing step 10000 out of: 1579584      1%
Writing step 20000 out of: 1579584      1%
Writing step 30000 out of: 1579584      2%
Writing step 40000 out of: 1579584      3%
Writing step 50000 out of: 1579584      3%
Writing step 60000 out of: 1579584      4%
Writing step 70000 out of: 1579584      4%
Writing step 80000 out of: 1579584      5%
Writing step 90000 out of: 1579584      6%
Writing step 100000 out of: 1579584     6%
Writing step 110000 out of: 1579584     7%
Writing step 120000 out of: 1579584     8%
Writing step 130000 out of: 1579584     8%
Writing step 140000 out of: 1579584     9%
Writing step 150000 out of: 1579584     9%
Writing step 160000 out of: 1579584     10%
Writing step 170000 out of: 1579584     11%
Writing step 180000 out of: 1579584     11%
Writing step 190000 out of: 1579584     12%
Writing step 200000 out of: 1579584     13%
Writing step 210000 out of: 1579584     13%
Writing step 220000 out of: 1579584     14%
Writing step 230000 out of: 1579584     15%
Writing step 240000 out of: 1579584     15%
Writing step 250000 out of: 1579584     16%
Writing step 260000 out of: 1579584     16%
Writing step 270000 out of: 1579584     17%
Writing step 280000 out of: 1579584     18%
Writing step 290000 out of: 1579584     18%
Writing step 300000 out of: 1579584     19%
Writing step 310000 out of: 1579584     20%
Writing step 320000 out of: 1579584     20%
Writing step 330000 out of: 1579584     21%
Writing step 340000 out of: 1579584     22%
Writing step 350000 out of: 1579584     22%
Writing step 360000 out of: 1579584     23%
Writing step 370000 out of: 1579584     23%
Writing step 380000 out of: 1579584     24%
Writing step 390000 out of: 1579584     25%
Writing step 400000 out of: 1579584     25%
Writing step 410000 out of: 1579584     26%
Writing step 420000 out of: 1579584     27%
Writing step 430000 out of: 1579584     27%
Writing step 440000 out of: 1579584     28%
Writing step 450000 out of: 1579584     28%
Writing step 460000 out of: 1579584     29%
Writing step 470000 out of: 1579584     30%
Writing step 480000 out of: 1579584     30%
Writing step 490000 out of: 1579584     31%
Writing step 500000 out of: 1579584     32%
Writing step 510000 out of: 1579584     32%
Writing step 520000 out of: 1579584     33%
Writing step 530000 out of: 1579584     34%
Writing step 540000 out of: 1579584     34%
Writing step 550000 out of: 1579584     35%
Writing step 560000 out of: 1579584     35%
Writing step 570000 out of: 1579584     36%
Writing step 580000 out of: 1579584     37%
Writing step 590000 out of: 1579584     37%
Writing step 600000 out of: 1579584     38%
Writing step 610000 out of: 1579584     39%
Writing step 620000 out of: 1579584     39%
Writing step 630000 out of: 1579584     40%
Writing step 640000 out of: 1579584     41%
Writing step 650000 out of: 1579584     41%
Writing step 660000 out of: 1579584     42%
Writing step 670000 out of: 1579584     42%
Writing step 680000 out of: 1579584     43%
Writing step 690000 out of: 1579584     44%
Writing step 700000 out of: 1579584     44%
Writing step 710000 out of: 1579584     45%
Writing step 720000 out of: 1579584     46%
Writing step 730000 out of: 1579584     46%
Writing step 740000 out of: 1579584     47%
Writing step 750000 out of: 1579584     47%
Writing step 760000 out of: 1579584     48%
Writing step 770000 out of: 1579584     49%
Writing step 780000 out of: 1579584     49%
Writing step 790000 out of: 1579584     50%
Writing step 800000 out of: 1579584     51%
Writing step 810000 out of: 1579584     51%
Writing step 820000 out of: 1579584     52%
Writing step 830000 out of: 1579584     53%
Writing step 840000 out of: 1579584     53%
Writing step 850000 out of: 1579584     54%
Writing step 860000 out of: 1579584     54%
Writing step 870000 out of: 1579584     55%
Writing step 880000 out of: 1579584     56%
Writing step 890000 out of: 1579584     56%
Writing step 900000 out of: 1579584     57%
Writing step 910000 out of: 1579584     58%
Writing step 920000 out of: 1579584     58%
Writing step 930000 out of: 1579584     59%
Writing step 940000 out of: 1579584     60%
Writing step 950000 out of: 1579584     60%
Writing step 960000 out of: 1579584     61%
Writing step 970000 out of: 1579584     61%
Writing step 980000 out of: 1579584     62%
Writing step 990000 out of: 1579584     63%
Writing step 1000000 out of: 1579584    63%
Writing step 1010000 out of: 1579584    64%
Writing step 1020000 out of: 1579584    65%
Writing step 1030000 out of: 1579584    65%
Writing step 1040000 out of: 1579584    66%
Writing step 1050000 out of: 1579584    66%
Writing step 1060000 out of: 1579584    67%
Writing step 1070000 out of: 1579584    68%
Writing step 1080000 out of: 1579584    68%
Writing step 1090000 out of: 1579584    69%
Writing step 1100000 out of: 1579584    70%
Writing step 1110000 out of: 1579584    70%
Writing step 1120000 out of: 1579584    71%
Writing step 1130000 out of: 1579584    72%
Writing step 1140000 out of: 1579584    72%
Writing step 1150000 out of: 1579584    73%
Writing step 1160000 out of: 1579584    73%
Writing step 1170000 out of: 1579584    74%
Writing step 1180000 out of: 1579584    75%
Writing step 1190000 out of: 1579584    75%
Writing step 1200000 out of: 1579584    76%
Writing step 1210000 out of: 1579584    77%
Writing step 1220000 out of: 1579584    77%
Writing step 1230000 out of: 1579584    78%
Writing step 1240000 out of: 1579584    79%
Writing step 1250000 out of: 1579584    79%
Writing step 1260000 out of: 1579584    80%
Writing step 1270000 out of: 1579584    80%
Writing step 1280000 out of: 1579584    81%
Writing step 1290000 out of: 1579584    82%
Writing step 1300000 out of: 1579584    82%
Writing step 1310000 out of: 1579584    83%
Writing step 1320000 out of: 1579584    84%
Writing step 1330000 out of: 1579584    84%
Writing step 1340000 out of: 1579584    85%
Writing step 1350000 out of: 1579584    85%
Writing step 1360000 out of: 1579584    86%
Writing step 1370000 out of: 1579584    87%
Writing step 1380000 out of: 1579584    87%
Writing step 1390000 out of: 1579584    88%
Writing step 1400000 out of: 1579584    89%
Writing step 1410000 out of: 1579584    89%
Writing step 1420000 out of: 1579584    90%
Writing step 1430000 out of: 1579584    91%
Writing step 1440000 out of: 1579584    91%
Writing step 1450000 out of: 1579584    92%
Writing step 1460000 out of: 1579584    92%
Writing step 1470000 out of: 1579584    93%
Writing step 1480000 out of: 1579584    94%
Writing step 1490000 out of: 1579584    94%
Writing step 1500000 out of: 1579584    95%
Writing step 1510000 out of: 1579584    96%
Writing step 1520000 out of: 1579584    96%
Writing step 1530000 out of: 1579584    97%
Writing step 1540000 out of: 1579584    97%
Writing step 1550000 out of: 1579584    98%
Writing step 1560000 out of: 1579584    99%
Writing step 1570000 out of: 1579584    99%
SYMPTOMDATA.csv Written

如果需要额外的信息，请告诉我。

csv

来源：https://stackoverflow.com/questions/77224475/buffered-writer-stopping-at-4000

1条答案

按热度按时间

gc0ot86w1#

我的问题原来是基于我输入的数组毕竟这是不清楚的，因为我是计数的步骤完成，而不是计数的实际次数，它实际上写的东西到文件。
这是一个更好的测试方法

private static void writeSortedFile(String outputPath, int[] arr) {
        DecimalFormat dec = new DecimalFormat("#%");

        try (FileWriter fileWriter = new FileWriter(outputPath);) {
            BufferedWriter bufferedWriter = new BufferedWriter(fileWriter);

            // Write header line to the CSV file
            String header = "";
            header = "VAERS_ID, AGE_YRS, SEX, RPT_Date, DIED, DATEDIED, SYMPTOM_TEXT, VAX_LIST, SYMPTOM_LIST";
            bufferedWriter.write(header);
            bufferedWriter.newLine();

            int counter = 0;
            for (int i = 0; i < arr.length; i++) {
                
                if (map.get(arr[i]) != null) {
                    
                    if (counter % 50000 == 0) {
                    System.out.println("Writing step " + counter + " out of: " + arr.length + "\t"
                            + dec.format(((double) counter / (double) arr.length)));
                    }
                    counter++;
                    Patient v = map.get(arr[i]);
                    String age = String.valueOf(v.getAGE_YRS());
                    String ID = String.valueOf(arr[i]);
                    String vaxList = " ";
                    String symptomList = " ";

                    for (Vaccine vax : v.getVAX_LIST()) {
                        vaxList += vax.toString();
                    }
                    for (Symptom symptom : v.getSYMPTOM_LIST()) {
                        symptomList += symptom.toString();
                    }

                    String[] data = { ID, age, v.getSEX(), v.getRPT_DATE(), v.getDIED(), v.getDATEDIED(),
                            v.getSYMPTOM_TEXT(), vaxList, symptomList };
                    String dataLine = String.join(",", data);
                    try {
                        bufferedWriter.write(dataLine);
                        bufferedWriter.newLine();
                    } catch (IOException e) {
                        e.printStackTrace();
                    } finally {
                        bufferedWriter.flush();
                    }
                    
                }
            }

            System.out.println(outputPath + " Written \n");
            bufferedWriter.close();

        } catch (IOException e) {
            e.printStackTrace();
            System.out.println("CSV Writing Error");
        }
    }

赞(0）回复(0）举报 2023-10-13

我来回答

csv Buffered writer stopping at 4000 [关闭]

1条答案

相关问题

热门标签

最新问答