- Writer
- DictWriter
We are going to use these CSV writing methods to append our newly created data to an existing CSV file.
Method 1: Using CSV Writer () Method
In this method, we are going to use the writer () method of the CSV to append the newly created data to the CSV.
CSV File: The below CSV file (test.csv) we will be used to append new row.
JAN,340,360,417
FEB,318,342,391
MAR,362,406,419
APR,348,396,461
JAN,340,360,417
FEB,318,342,391
from csv import writer
# this function which append new row data into an existing csv file
def appendNewRow(csvFileName, elementsToAppend):
# open the csv file in the append mode
with open(csvFileName, 'a+', newline='') as append_obj:
# created a writer object from the writer module
append_writer = writer(append_obj)
# the created writer object which writes new row to the csv
append_writer.writerow(elementsToAppend)
# list of new elements which we want to append
newrow = ["DEC", 337, 405, 432]
# calling a function which accepts two parameters csv file and new row data
appendNewRow('test.csv', newrow)
Output:
JAN,340,360,417
FEB,318,342,391
MAR,362,406,419
APR,348,396,461
JAN,340,360,417
FEB,318,342,391
DEC,337,405,432
Line 1: We import the writer module.
Line 4 to 10: We created a function named “appendNewRow” which accepts two parameters (the name of the existing CSV file and the data). We open the CSV file (existing CSV file) with append mode (‘a’), so that we can add our new data to the CSV file. For that, we created an object (append_writer) from the writer module. And in the next line, we are calling method writerow from the append_writer object to append the data to the CSV.
Line 13 and 15: We created a new data list which we want to append to an existing CSV file. In line 15, we are calling the function “appendNewRow” and passing two parameters (the name of the existing CSV file and the data).
Now, we are going to append data where some data is missing.
In the previous example, we have seen there’s no missing column value in the newly created data. But what will happen if we have some missing column value? So, even if we have some missing column value, the writer module of the CSV has no method to check either all columns values are present in the data or not.
newrow = ["DEC", 337, 432]
# calling a function which accepts two parameters csv file and new row data
appendNewRow('test.csv', newrow)
Output:
JAN,340,360,417
FEB,318,342,391
MAR,362,406,419
APR,348,396,461
JAN,340,360,417
FEB,318,342,391
DEC,337,405,432
DEC,337,432
We created a new dataset where one column value is missing. So when we run the above example program, it gives a simple appended result as shown in the above output. But in the output itself, we can see that the value 405 is missing. As a result, the CSV writer module lacks a method for determining any missing column value.
So, if we want to adjust the proper column wise value of the newly created data, even if there are some missing values, we must pass some empty string value along with the data, like below:
The above case is not a big deal because it is just a single row of data, but when we have billions of rows, we can’t handle such situations with the empty string adding method. Because then it will become a very tedious job.
Method 2: Using DictWriter () Method
In this method, we are going to discuss the DictWriter () method to append the newly created data to the csv. The DictWriter () method also has the facility to figure out which column value is missing because it is a dictionary-based method and if any key value is empty, it will keep empty string automatically.
CSV File: We are using the same CSV file (test.csv) as we have used in the previous examples.
def appendNewRow(csvFileName, elementsToAppend):
# open the csv file in the append mode
with open(csvFileName, 'a+', newline='') as append_obj:
# created a writer object from the writer module
append_writer = DictWriter(append_obj,
fieldnames = ['Month', '1958', '1959','1960'])
# the created writer object which writes new row to the csv
append_writer.writerow(elementsToAppend)
# list of new elements which we want to append
newrow = {"Month":"DEC", "1958":"337", "1959":"405", "1960" : "432" }
# calling a function which accepts two parameters csv file and new row data
appendNewRow('test.csv', newrow)
Output:
JAN,340,360,417
FEB,318,342,391
MAR,362,406,419
APR,348,396,461
JAN,340,360,417
FEB,318,342,391
DEC,337,405,432
Line 1: We import the DictWriter method.
Line 4 to 12: We created a function named “appendNewRow” which accepts two parameters (the name of the existing CSV file and the data). We open the csv file (existing) with append mode (‘a’), so that we can add our new dict data to the CSV file. Then we pass the header of the CSV in the form of a list to the DictWriter while creating the object itself. We have to tell the column names of the csv to the object before writing to the csv. Otherwise, it will generate an error because the object will not be able to understand the key names of the dictionary. And in the next line, we are calling method writerow from the append_writer object to append the dictionary data to the CSV.
Line 14 to 16: We created a new data dictionary which we want to append to an existing CSV file. In line 16, we are calling the function “appendNewRow” and passing two parameters (the name of the existing CSV file and the data).
Now, we are going to append data where some data is missing.
As we know, the main problem with the writer method is that there is no method to automatically detect that any column value is missing. But the DictWriter method can guess this automatically with the help of the missing keys. So let’s explore this with an example.
newrow = {"Month":"DEC", "1958":"337", "1960" : "432" }
# calling a function which accepts two parameters csv file and new row data
appendNewRow('test.csv', newrow)
Now, we are going to pass new dict data where the key (‘1959’) with its value is missing. So, let’s pass this data to the programme and see the result.
Output:
JAN,340,360,417
FEB,318,342,391
MAR,362,406,419
APR,348,396,461
JAN,340,360,417
FEB,318,342,391
DEC,337,405,432
DEC,337,,432
The above output shows that in place of the missing value, the DictWriter method put an empty string.
Conclusion
We have seen two methods to write newly created data to an existing CSV file. The first method writer is good, but only in that case when we don’t have any missing column values. Because, as we have already seen, the writer method of CSV is not able to handle missing value situations, and it will enter a particular column value into some other column value. So, this is a major drawback of the writer method. But this situation is very easily handled by the DictWriter method because of the keys.
