SammMcCandless
New Member
- Joined
- May 17, 2024
- Messages
- 2
- Office Version
- 365
- Platform
- Windows
Hi all,
I have several data files in a standard format (xlsx files, 1 tab (all named the same), 4 columns of data each (all common headers(Owner, Supplier, Amount and Account)) - varying numbers of rows in each. These files are in a folder on sharepoint (no option to save these outside of here for this piece of work). All files together will be around 400k rows of data.
I have used power query to get data from this folder, and combine the files, then perform some further cleaning on the data, however the refresh rate is so slow (5+ hours before crashing out (I left it runnign one night just in case!)) that it is unusable. The same M code works on a much smaller sample (2 files of same format rather than 35, and fewer rows in each) so assume its not an error as such, but wondering if the slow rate is due to getting data from sharepoint, or something else I'm trying to do!
Any help greatly appreciated. Some screenshots/examples below (source data adv. edit is the file with the main work in it), but let me know if more info needed. Thanks
(ps. if I'm using list.buffer and table.buffer incorrectly, I will gladly take any tips, but the issue exsisted before including these, so taking out might not solve my issue)
I have several data files in a standard format (xlsx files, 1 tab (all named the same), 4 columns of data each (all common headers(Owner, Supplier, Amount and Account)) - varying numbers of rows in each. These files are in a folder on sharepoint (no option to save these outside of here for this piece of work). All files together will be around 400k rows of data.
I have used power query to get data from this folder, and combine the files, then perform some further cleaning on the data, however the refresh rate is so slow (5+ hours before crashing out (I left it runnign one night just in case!)) that it is unusable. The same M code works on a much smaller sample (2 files of same format rather than 35, and fewer rows in each) so assume its not an error as such, but wondering if the slow rate is due to getting data from sharepoint, or something else I'm trying to do!
Any help greatly appreciated. Some screenshots/examples below (source data adv. edit is the file with the main work in it), but let me know if more info needed. Thanks
(ps. if I'm using list.buffer and table.buffer incorrectly, I will gladly take any tips, but the issue exsisted before including these, so taking out might not solve my issue)
