Importing data from Folder through Power Query

earthworm

Well-known Member
Joined
May 19, 2009
Messages
766
Office Version
  1. 2019
  2. 2016
Platform
  1. Windows
I have 30 files . Each files contains data of approx 6 Million. I imported the folder through Power query. Then I get and transform data and then I used the formula Excel.Workbook(Content). When I am expanding the column power query is taking too much time in loading file before I move to next step of expanding the column to show data. Is there any faster way ? Because I have to wait for hours before moving to next step. Please advice
 

Excel Facts

Bring active cell back into view
Start at A1 and select to A9999 while writing a formula, you can't see A1 anymore. Press Ctrl+Backspace to bring active cell into view.
What file format are the source files?
Are the files/folder on a network drive? If so, move them to your local drive and it should be a lot better.
 
Upvote 0
You say each file contains "approx 6 million". 6 million what? It can't be rows. When you say "I used the formula Excel.Workbook(Content)" did you do that manually, or did you use the UI to combine the files automatically?
 
Upvote 0
You say each file contains "approx 6 million". 6 million what? It can't be rows. When you say "I used the formula Excel.Workbook(Content)" did you do that manually, or did you use the UI to combine the files automatically?
6 Million Rows. I used the formula manually
 
Upvote 0
You are not telling me something. You say your source files are Excel files and you say they contain more than 6 million rows. An excel sheet has a maximum of 1 million rows, so something else is going on. Are the 6 million rows spread across more than 6 sheets in the same book?
 
Upvote 0
You are not telling me something. You say your source files are Excel files and you say they contain more than 6 million rows. An excel sheet has a maximum of 1 million rows, so something else is going on. Are the 6 million rows spread across more than 6 sheets in the sa

You are not telling me something. You say your source files are Excel files and you say they contain more than 6 million rows. An excel sheet has a maximum of 1 million rows, so something else is going on. Are the 6 million rows spread across more than 6 sheets in the same book?

I need a merge a data from each excel file that contains 800,000 on each excel file
 
Upvote 0
Here are some thoughts:
  1. are you merging or appending? Appending is generally faster than merging but you may not have a choice
  2. I suggest you use the UI, not manually written queries. Usually the UI knows best
  3. You say it is "too slow" but you don't specify how long it is taking. I would expect such a query to take about 3-5 mins assuming pure append.
  4. What is the file extension? if it is XLSX it will probably be faster than XLSB
 
Upvote 0

Forum statistics

Threads
1,216,499
Messages
6,131,012
Members
449,613
Latest member
MedDash99

We've detected that you are using an adblocker.

We have a great community of people providing Excel help here, but the hosting costs are enormous. You can help keep this site running by allowing ads on MrExcel.com.
Allow Ads at MrExcel

Which adblocker are you using?

Disable AdBlock

Follow these easy steps to disable AdBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the icon in the browser’s toolbar.
2)Click on the "Pause on this site" option.
Go back

Disable AdBlock Plus

Follow these easy steps to disable AdBlock Plus

1)Click on the icon in the browser’s toolbar.
2)Click on the toggle to disable it for "mrexcel.com".
Go back

Disable uBlock Origin

Follow these easy steps to disable uBlock Origin

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back

Disable uBlock

Follow these easy steps to disable uBlock

1)Click on the icon in the browser’s toolbar.
2)Click on the "Power" button.
3)Click on the "Refresh" button.
Go back
Back
Top