How to export more than 1 million rows to Excel with SSIS 2016?

kokeuurv  于 2023-02-21  发布在  其他
关注(0)|答案(1)|浏览(158)

I'm using SSIS with SQL Server 2016 to produce both text and Excel files (also version 2016). Some data flow tasks return more than a million rows of results. Writing to text is not an issue. However, Excel is limited to 1 million rows per sheet.

How do I configure the Excel Destination, or other component, to ensure more than a million rows are saved in the target workbook using multiple sheets as needed?

bf1o4zei

bf1o4zei1#

Sometimes, you need to push back to whomever has requested "exporting all the data to Excel" as it's just not an option. Is an analyst really going to be able to do anything with a a spreadsheet with multi-million rows? No, no they will not.

Purely as an exercise of "how could I do this really bad thing"

(excluded from this exercise is a custom Script Destination as I didn't feel like writing code)

You must determine ahead of time what is a reasonable upper bound on the the number of sheets to be created. You can either limit your worksheet to a million or hit the actual limit of 1,048,576 rows per sheet. Possibly 1,048,575 rows because you want to repeat your header across sheets.

Whatever that maximum number of sheets is, N, you will need to create to create N Excel Destinations

You'll need to have a ROW_NUMBER() function applied to your source data so you'll have to have a custom query there

SELECT *, ROW_NUMBER() OVER (ORDER BY (SELECT NULL)) AS RowNum FROM dbo.MyTable;

The Conditional Split will use the modulo function to assign rows to their paths

  1. RowNum % N == 1
  2. RowNum % N == 2
  3. ...
  4. RowNum % N == (N-1)

and the default output path is for == 0

相关问题