90,259 questions
Score of -2
0 answers
98 views
Efficient header resolution strategy for zero-copy CSV parsing in C [closed]
I'm building a CSV ingestion engine in C11 using POSIX mmap() and zero-copy parsing.
The parser reads the header row once and maps required logical fields to their corresponding column indices so that ...
- reputation score 1
Score of 0
0 answers
39 views
loadTextEx reports "Number of columns is inconsistent" despite matching schema and target table columns
I'm using loadTextEx to import a CSV into a partitioned table. I generated the schema with extractTextSchema, which shows the same number of columns as the target table. But loadTextEx still throws a ...
- reputation score 1
Advice
1
vote
4
replies
124
views
How to display the contents of a CSV file, with rows filtered by the value in the timestamp column
Is it possible for a Python 3 script to:
Read a CSV file (and display a loading screen/progress indicator while doing so);
Only display rows/results where the 'status' column value is set to '...
- reputation score 1
Score of 0
0 answers
51 views
Import compressed TongLian data files directly or need to decompress first?
I'm working with compressed data files from TongLian (通联数据), delivered as .zip archives. I'm wondering if DolphinDB can load the data directly from the .zip file, or if I need to decompress before ...
- reputation score 1
Tooling
0
votes
1
replies
131
views
How can I read a CSV file, sum the “Betrag” column, count rows, and group totals by tenant?
I have a CSV file exported from Excel. It contains at least these two columns:
Betrag (numeric amounts)
tenant (string identifier)
I want to:
Read the CSV file
Calculate the total sum of the ...
- reputation score 1
Tooling
0
votes
5
replies
128
views
Comparison of Two CSV Files with a Large Number of Columns Spanning Multiple Lines (Generated from an SQL*Plus Spool)
I have these two CSV files and I have already tried using Copilot without success.
https://drive.google.com/file/d/1GURMVn10LCMQFHOVpgwcW-6SUnqDBO01/view?usp=drive_link
https://drive.google.com/file/d/...
- reputation score 1
Best practices
1
vote
15
replies
326
views
How do I properly handle UTF8 encoded CSV files
I have code that imports spreadsheets of 2 columns of numbers for a scientific application.
While creating some test files with excel, I inadvertently created one with "CSV UTF-8" encoding. ...
- reputation score 31
Best practices
0
votes
2
replies
134
views
How to incrementally merge daily CSV delta updates into a master file without losing expired records?
I am building a data pipeline to create a cumulative master dataset from daily CSV exports generated by a booking system.
Context & Data Structure
Source: A daily CSV file containing status ...
- reputation score 1
Best practices
0
votes
1
replies
68
views
CSV Column Structure Evolution Snowflake
I'm stuck with a client's request.
Context: Ingestion of CSV files from a S3 bucket to a table in Snowflake. In addition, I need to add three columns, one with the current date, one with the name of ...
- reputation score 21
Score of 0
0 answers
71 views
Azure Fabric - Lakehouse SQL endpoint - CSV with non UTF codepage
I'm trying to use OPENROWSET over a very simple CSV file with only 3 columns
col1;col2;col3
FRANK;998533805;20.06
FINSÅS KARL;999999;10.05
RØNNING FRANK KARL;999999;7.05
but since the encoding of the ...
- reputation score 21
Score of 3
3 answers
168 views
Write multi-columns to CSV with for loop
This code snippet contains bpy. I am trying to write data to a three column CSV. The first column is the vertex index, the second column is the U coordinate, and the third column is the V coordinate.
...
- reputation score 31
Score of 1
0 answers
64 views
CsvValidator optional header column
I am using the UK National Archives CSV Validator via the Maven dependency uk.gov.nationalarchives:csv-validator-java-api-1.4.0.jar to validate both the headers and the body of CSV/txt files.
I am ...
- reputation score 39
Score of 1
2 answers
133 views
download csv file from OSF
I am trying to download a .csv file from a Public Open Science Framework (OSF) project through my script on GitHub. I do not want to download a local copy.
I've tried using two different options for ...
- reputation score 1539
Score of 3
5 answers
251 views
Fusing row with previous row(s) depending on criteria
I'm trying to fix a CSV file I received in which actual new lines were inserted throughout the data, effectively splitting one row of data into multiple ones with incomplete columns. The line breaks ...
- reputation score 33
Score of 3
2 answers
205 views
Python polars script leaks memory and crashes (scans CSV to lazyframes - writes to database)
I wrote a small script to scan a CSV in Python polars, select specific columns/filter specific rows in lazyframes and upload the result to a Postgres DB.
The script works with a smaller test CSV but, ...
- reputation score 31
