Hari Nair
Python is a general-purpose interpreted, interactive, object-oriented programming language similar to Perl, C, and Java.
I am using Python 3.7.2. on my laptop. To check the version type
c:\python -V
Output
Python 3.7.2
Variable Types
Python variables do not need explicit declaration to reserve memory space. The equal sign = is used to assign values to variables.
a = b = c = 1
Standard Data types
Numbers:
- int (signed integers)
- long (long integers, they can also be represented in octal and hexadecimal)
- float (floating point real values)
- complex (complex numbers)
String:
Strings are identified as a contiguous set of characters represented in the quotation marks. Python allows for either pairs of single or double quotes.
#!/usr/bin/python
Example-1
str = 'Hello World!'
print (str) # complete string
print (str[0]) # first character of the string
print (str[2:5]) # characters starting from 3rd to 5th
print (str[2:]) # string starting from 3rd character
print (str * 2) # string two times
print (str + "hari") # concatenated string
Output:
Hello World!
H
llo
llo World!
Hello World!Hello World!
Hello World!hari
List:
A list contains items separated by commas and enclosed within square brackets ([]).
Example-2
list = [ 'abcd', 786 , 2.23, 'hari', 70.2 ]
tinylist = [123, 'hari nair']
print (list) # complete list
print (list[0]) # first element of the list
print (list[1:3]) # elements starting from 2nd till 3rd
print (list[2:] ) # elements starting from 3rd element
print (tinylist * 2) # list two times
print (list + tinylist) # Prints concatenated lists
Output
['abcd', 786, 2.23, 'hari', 70.2]
abcd
[786, 2.23]
[2.23, 'hari', 70.2]
[123, 'hari nair', 123, 'hari nair']
['abcd', 786, 2.23, 'hari', 70.2, 123, 'hari nair']
Tuple:
A tuple consists of a number of values separated by commas. Tuples are enclosed within parentheses.
It is a read-ony list
Advantages of implementing a tuple over a list
- Tuple is generally used with different datatypes and list for similar datatypes (write-protected).
- Tuple are immutable, iterating through tuple is faster
- Tuples that contain immutable elements can be used as key for a dictionary; with list this is not possible
my_tuple = (1, "Hari Nair", 1962, "India","Emergys Chennai")
print(my_tuple)
Dictionary:
Dictionaries are enclosed by curly braces ({ }) and values can be assigned and accessed using square braces ([]). Dictionaries have no concept of order among elements.
squares = {1: "Hari", 2: "Nair", 3: "Chennai", 7: "Emergys", 9: "India"}
for i in squares:
print(squares[i])
Output
Hari
Nair
Chennai
Emergys
India
for i in squares:
print(squares[i])
Output
Hari
Nair
Chennai
Emergys
India
Example-3
dict = {}
dict['one'] = "This is one"
dict[2] = "This is two"
tinydict = {'name': 'hari','employee_code':6734, 'dept': 'operations'}
print (dict['one']) # value for 'one' key
print (dict[2]) # value for 2 key
print (tinydict) # complete dictionary
print (tinydict.keys()) # all the keys
print (tinydict.values()) # all the values
Output:
This is one
This is two
{'name': 'hari', 'employee_code': 6734, 'dept': 'operations'}
dict_keys(['name', 'employee_code', 'dept'])
dict_values(['hari', 6734, 'operations'])
Date time
Example-4
import datetime
from datetime import date # note this usage
today=date.today()
print(today)
print (today.strftime("%A, %B, %d, %Y"))
Output
2019-02-09
Saturday, February, 09, 2019
If Else
Example-5
v1=3
v2=4
v3=5.3
name1='Hari Nair'
name2="My Name is " + name1
print (name2)
if v1==10: # : important!
print ("v1 is 100")
print ("v2 is 200")
elif v2==20:
print ("v2 is 200")
else:
print ("v1 not 100")
print ("Over")
print (v1*v2)
print (v1**v2) # to the power
print (v1*v1*v1*v1)
print (v1/v2)
print (v1//v2)
Output
My Name is Hari Nair
v1 not 100
Over
12
81
81
0.75
0
Example-6
for loop
for i in range(5):
print (i,i**2)
Output
0 0
1 1
2 4
3 9
4 16
Example-7
While loop
i=1
while i < 5:
print (i,i**2)
i+=1
Output
1 1
2 4
3 9
4 16
Example-8
Functions
def print_square(v1):
""" function to print square """
return (v1,v1**2)
print (print_square(5))
Output
(5, 25)
Example-9
Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language for reading and writing DATA between in-memory data structures and different formats: CSV and text files, Microsoft Excel, SQL databases, and HDF5. (Hierarchical Data Format HDF is a set of file formats (HDF4, HDF5) designed to store and organize large amounts of data).
import pandas as pd
data = [['Hari',60],['Baby',21],['Jolly',3]]
df = pd.DataFrame(data,columns=['Name','Age'])
print (df)
Output:
Name Age
0 Hari 60
1 Baby 21
2 Jolly 3
import pandas as pd
d = {'one' : pd.Series([1, 2, 3], index=['a', 'b', 'c']),
'two' : pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])}
df = pd.DataFrame(d)
# Adding a new column to an existing DataFrame object with column label by passing new series
print ("Adding a new column by passing as Series:")
df['three']=pd.Series([10,20,30],index=['a','b','c'])
print (df)
print ("Adding a new column using the existing columns in DataFrame:")
df['four']=df['one']+df['three']
print (df)
Output:
Adding a new column by passing as Series:
one two three
a 1.0 1 10.0
b 2.0 2 20.0
c 3.0 3 30.0
d NaN 4 NaN
Adding a new column using the existing columns in DataFrame:
one two three four
a 1.0 1 10.0 11.0
b 2.0 2 20.0 22.0
c 3.0 3 30.0 33.0
d NaN 4 NaN NaN
#Example-9.py
# Reading excel files using Python
# Python -m pip install xlrd
import pandas as pd
data = pd.read_excel('sales.xls')
print(data.head())
Output
Year Month Area Sales Cogs Profit
0 2017 Jan Delhi 4.1 3.70 0.40
1 2017 Feb Mumbai 4.0 3.60 0.40
2 2017 Mar Tamilnadu 3.8 3.65 0.15
3 2017 Apr AP 4.3 4.00 0.30
4 2017 May MP 5.1 4.80 0.30
Dask is a parallel computing library . Dask is a useful tool in the data pipeline process. Dask provides advanced parallelism for analytics, enabling performance at scale. Dask can process data that doesn’t fit into memory by breaking it into blocks and specifying task chains. Dask can scale down to your laptop and up to a cluster. Here, we'll use an environment you setup on your laptop to analyze medium sized datasets in parallel locally.
# glob1.py
from sklearn.datasets import make_classification
import pandas as pd
for i in range(1, 10):
print('Generating trainset %d' % i)
x, y = make_classification(n_samples=100_000, n_features=100)
df = pd.DataFrame(data=x)
df['y'] = y
df.to_csv('trainset_%d.csv' % i, index=False)
import glob
df_list = []
for filename in glob.glob('trainset_*.csv'):
df_ = pd.read_csv(filename)
df_list.append(df_)
df = pd.concat(df_list)
print (df.shape)
Database
PostgreSQL and MySQL are two of the most common open source databases for storing Python web applications' data. SQLite is a database that is stored in a single file on disk. SQLite is built into Python but is only built for access by a single connection at a time.
Python with MySQL database
Installing library mysql-connector
C:\Users\HaridasanNair>python -m pip install mysql-connector
Collecting mysql-connector
Downloading https://files.pythonhosted.org/packages/59/e0/775bf5fb3dd4c7f9aa6877907d4a96eecca6886c603dedfea6e843e94560/mysql-connector-2.1.6.tar.gz (11.8MB)
17% |█████▌ | 2.0MB 1.4MB/s eta 0:00:07
>>>
Example-10
#MySql database - insert records
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="yourusername",
passwd="yourpassword"
)
mycursor = mydb.cursor()
sql = "INSERT INTO customers (name, address) VALUES (%s, %s)"
val = ("Hari Nair", "Chennai, TN")
mycursor.execute(sql, val)
mydb.commit()
print(mycursor.rowcount, "record inserted.")
Output
1 record inserted.
Example-11
print ('%*.*f' % (4, 2, 2.71182)) # 4 width of printed number 2 decimal places rounded
print ('%*.*f' % (6, 2, 2.789182)) # 6 width of printed number 2 decimal places rounded
print ('%*.*f' % (4, 2, 56.871)) # 4 width of printed number 2 decimal places rounded
2.79
56.87
Example-12
#Python has an in-built support for SQlite
import sqlite3
conn = sqlite3.connect('database.db')
#conn.execute('drop table m_customers')
conn.execute('CREATE TABLE m_customers (cust_id TEXT, cust_name TEXT, cust_address TEXT,cust_city TEXT, cust_pin numeric)')
print ("Table created successfully");
conn.close()
Example-13
#Python - insert data
import sqlite3
conn = sqlite3.connect('database.db')
#conn.execute('drop table t_gl')
#conn.execute ("delete from t_gl")
#conn.execute ('CREATE TABLE t_gl (sysid INTEGER, gl_date DATE,gl_txnno INTEGER,gl_descr TEXT, gl_anal1 TEXT, gl_anal2 TEXT,gl_DrAc TEXT, gl_CrAc TEXT , gl_ref TEXT ,gl_amount NUMERIC)')
conn.execute ("INSERT INTO t_gl VALUES (1,'02/01/2019',190001,'Salary paid Hari','','','Salary A/c','To Cash','',1000)")
conn.execute ("INSERT INTO t_gl VALUES (2,'02/01/2019',190001,'Salary paid Kumari','','','Salary A/c','To Cash','',9000)")
conn.execute ("INSERT INTO t_gl VALUES (3,'02/01/2019',190001,'Salary paid Kavita','','','Salary A/c','To Cash','',11000)")
conn.execute ("INSERT INTO t_gl VALUES (4,'02/01/2019',190001,'Salary paid Meena','','','Salary A/c','To Cash','',12000)")
print ("General ledger created successfully");
conn.close()
Reflection in Python
Reflection is the ability for code to be able to examine attributes about objects that might be passed as parameters to a function. For example, if we write type(obj) then Python will return an object which represents the type of obj.
Example-14
#ref1.py
# Python program to illustrate reflection
def reverse(seq):
SeqType = type(seq)
emptySeq = SeqType()
if seq == emptySeq:
return emptySeq
restrev = reverse(seq[1:])
first = seq[0:1]
# Combine the result
result = restrev + first
return result
# Driver code
print(reverse([1, 2, 3, 4]))
print(reverse("Hari nair"))
Output
[4, 3, 2, 1]
rian iraH
Little sound is always good. Se a Text to speech code
import library pyglet and gtts
Example-15
#sp1.py
#
# Import the required module for text to speech conversion
from gtts import gTTS
# This module is imported so that we can
# play the converted audio
import os
import pyglet
# The text that you want to convert to audio
mytext = 'Welcome to Emergys!, Your reporting manager is Python 3.7'
# Language in which you want to convert
language = 'en'
# Passing the text and language to the engine,
# here we have marked slow=False. Which tells
# the module that the converted audio should
# have a high speed
myobj = gTTS(text=mytext, lang=language, slow=False)
# Saving the converted audio in a mp3 file named
# welcome
myobj.save("welcome.mp3")
# Playing the converted file
play1 = pyglet.media.load('welcome.mp3')
play1.play()
pyglet.app.run()
play1.exit()
Example-15
# Handwritten numbers using sklearn
#
import matplotlib.pyplot as plt
from sklearn import datasets, svm, metrics
digits = datasets.load_digits()
images_and_labels = list(zip(digits.images, digits.target))
for index, (image, label) in enumerate(images_and_labels[:4]):
plt.subplot(2, 4, index + 1)
plt.axis('off')
plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
plt.title('Training: %i' % label)
n_samples = len(digits.images)
data = digits.images.reshape((n_samples, -1))
classifier = svm.SVC(gamma=0.001)
classifier.fit(data[:n_samples // 2], digits.target[:n_samples // 2])
expected = digits.target[n_samples // 2:]
predicted = classifier.predict(data[n_samples // 2:])
print("Classifier %s:\n%s\n"
% (classifier, metrics.classification_report(expected, predicted)))
print("Confusion matrix:\n%s" % metrics.confusion_matrix(expected, predicted))
images_and_predictions = list(zip(digits.images[n_samples // 2:], predicted))
for index, (image, prediction) in enumerate(images_and_predictions[:4]):
plt.subplot(2, 4, index + 5)
plt.axis('off')
plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
plt.title('Prediction: %i' % prediction)
plt.show()
Output
Classifier SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
decision_function_shape='ovr', degree=3, gamma=0.001, kernel='rbf',
max_iter=-1, probability=False, random_state=None, shrinking=True,
tol=0.001, verbose=False):
precision recall f1-score support
0 1.00 0.99 0.99 88
1 0.99 0.97 0.98 91
2 0.99 0.99 0.99 86
3 0.98 0.87 0.92 91
4 0.99 0.96 0.97 92
5 0.95 0.97 0.96 91
6 0.99 0.99 0.99 91
7 0.96 0.99 0.97 89
8 0.94 1.00 0.97 88
9 0.93 0.98 0.95 92
micro avg 0.97 0.97 0.97 899
macro avg 0.97 0.97 0.97 899
weighted avg 0.97 0.97 0.97 899
Confusion matrix:
[[87 0 0 0 1 0 0 0 0 0]
[ 0 88 1 0 0 0 0 0 1 1]
[ 0 0 85 1 0 0 0 0 0 0]
[ 0 0 0 79 0 3 0 4 5 0]
[ 0 0 0 0 88 0 0 0 0 4]
[ 0 0 0 0 0 88 1 0 0 2]
[ 0 1 0 0 0 0 90 0 0 0]
[ 0 0 0 0 0 1 0 88 0 0]
[ 0 0 0 0 0 0 0 0 88 0]
[ 0 0 0 1 0 1 0 0 0 90]]
Vaex is a python library to visualize and explore Big tabular datas. It can calculate statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid up to a billion rows per second.
Installing vaex
pip install vaex-core vaex-viz vaex-jupyter vaex-arrow vaex-server vaex-ui vaex-hdf5 vaex-astro vaex-distributed
Pandas is an open source library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language for reading and writing DATA between in-memory data structures and different formats: CSV and text files, Microsoft Excel, SQL databases, and HDF5. (Hierarchical Data Format HDF is a set of file formats (HDF4, HDF5) designed to store and organize large amounts of data).
import pandas as pd
data = [['Hari',60],['Baby',21],['Jolly',3]]
df = pd.DataFrame(data,columns=['Name','Age'])
print (df)
Output:
Name Age
0 Hari 60
1 Baby 21
2 Jolly 3
import pandas as pd
d = {'one' : pd.Series([1, 2, 3], index=['a', 'b', 'c']),
'two' : pd.Series([1, 2, 3, 4], index=['a', 'b', 'c', 'd'])}
df = pd.DataFrame(d)
# Adding a new column to an existing DataFrame object with column label by passing new series
print ("Adding a new column by passing as Series:")
df['three']=pd.Series([10,20,30],index=['a','b','c'])
print (df)
print ("Adding a new column using the existing columns in DataFrame:")
df['four']=df['one']+df['three']
print (df)
Output:
Adding a new column by passing as Series:
one two three
a 1.0 1 10.0
b 2.0 2 20.0
c 3.0 3 30.0
d NaN 4 NaN
Adding a new column using the existing columns in DataFrame:
one two three four
a 1.0 1 10.0 11.0
b 2.0 2 20.0 22.0
c 3.0 3 30.0 33.0
d NaN 4 NaN NaN
#Example-9.py
# Reading excel files using Python
# Python -m pip install xlrd
import pandas as pd
data = pd.read_excel('sales.xls')
print(data.head())
Output
Year Month Area Sales Cogs Profit
0 2017 Jan Delhi 4.1 3.70 0.40
1 2017 Feb Mumbai 4.0 3.60 0.40
2 2017 Mar Tamilnadu 3.8 3.65 0.15
3 2017 Apr AP 4.3 4.00 0.30
4 2017 May MP 5.1 4.80 0.30
Dask is a parallel computing library . Dask is a useful tool in the data pipeline process. Dask provides advanced parallelism for analytics, enabling performance at scale. Dask can process data that doesn’t fit into memory by breaking it into blocks and specifying task chains. Dask can scale down to your laptop and up to a cluster. Here, we'll use an environment you setup on your laptop to analyze medium sized datasets in parallel locally.
# glob1.py
from sklearn.datasets import make_classification
import pandas as pd
for i in range(1, 10):
print('Generating trainset %d' % i)
x, y = make_classification(n_samples=100_000, n_features=100)
df = pd.DataFrame(data=x)
df['y'] = y
df.to_csv('trainset_%d.csv' % i, index=False)
import glob
df_list = []
for filename in glob.glob('trainset_*.csv'):
df_ = pd.read_csv(filename)
df_list.append(df_)
df = pd.concat(df_list)
print (df.shape)
PostgreSQL and MySQL are two of the most common open source databases for storing Python web applications' data. SQLite is a database that is stored in a single file on disk. SQLite is built into Python but is only built for access by a single connection at a time.
Python with MySQL database
Installing library mysql-connector
C:\Users\HaridasanNair>python -m pip install mysql-connector
Collecting mysql-connector
Downloading https://files.pythonhosted.org/packages/59/e0/775bf5fb3dd4c7f9aa6877907d4a96eecca6886c603dedfea6e843e94560/mysql-connector-2.1.6.tar.gz (11.8MB)
17% |█████▌ | 2.0MB 1.4MB/s eta 0:00:07
>>>
Example-10
#MySql database - insert records
import mysql.connector
mydb = mysql.connector.connect(
host="localhost",
user="yourusername",
passwd="yourpassword"
)
mycursor = mydb.cursor()
sql = "INSERT INTO customers (name, address) VALUES (%s, %s)"
val = ("Hari Nair", "Chennai, TN")
mycursor.execute(sql, val)
mydb.commit()
print(mycursor.rowcount, "record inserted.")
Output
1 record inserted.
Example-11
print ('%*.*f' % (4, 2, 2.71182)) # 4 width of printed number 2 decimal places rounded
print ('%*.*f' % (6, 2, 2.789182)) # 6 width of printed number 2 decimal places rounded
print ('%*.*f' % (4, 2, 56.871)) # 4 width of printed number 2 decimal places rounded
Output
2.712.79
56.87
Example-12
#Python has an in-built support for SQlite
import sqlite3
conn = sqlite3.connect('database.db')
#conn.execute('drop table m_customers')
conn.execute('CREATE TABLE m_customers (cust_id TEXT, cust_name TEXT, cust_address TEXT,cust_city TEXT, cust_pin numeric)')
print ("Table created successfully");
conn.close()
Example-13
#Python - insert data
import sqlite3
conn = sqlite3.connect('database.db')
#conn.execute('drop table t_gl')
#conn.execute ("delete from t_gl")
#conn.execute ('CREATE TABLE t_gl (sysid INTEGER, gl_date DATE,gl_txnno INTEGER,gl_descr TEXT, gl_anal1 TEXT, gl_anal2 TEXT,gl_DrAc TEXT, gl_CrAc TEXT , gl_ref TEXT ,gl_amount NUMERIC)')
conn.execute ("INSERT INTO t_gl VALUES (1,'02/01/2019',190001,'Salary paid Hari','','','Salary A/c','To Cash','',1000)")
conn.execute ("INSERT INTO t_gl VALUES (2,'02/01/2019',190001,'Salary paid Kumari','','','Salary A/c','To Cash','',9000)")
conn.execute ("INSERT INTO t_gl VALUES (3,'02/01/2019',190001,'Salary paid Kavita','','','Salary A/c','To Cash','',11000)")
conn.execute ("INSERT INTO t_gl VALUES (4,'02/01/2019',190001,'Salary paid Meena','','','Salary A/c','To Cash','',12000)")
print ("General ledger created successfully");
conn.close()
Reflection in Python
Reflection is the ability for code to be able to examine attributes about objects that might be passed as parameters to a function. For example, if we write type(obj) then Python will return an object which represents the type of obj.
Example-14
# Python program to illustrate reflection
def reverse(seq):
SeqType = type(seq)
emptySeq = SeqType()
if seq == emptySeq:
return emptySeq
restrev = reverse(seq[1:])
first = seq[0:1]
# Combine the result
result = restrev + first
return result
# Driver code
print(reverse([1, 2, 3, 4]))
print(reverse("Hari nair"))
Output
[4, 3, 2, 1]
rian iraH
Little sound is always good. Se a Text to speech code
import library pyglet and gtts
Example-15
#sp1.py
#
# Import the required module for text to speech conversion
from gtts import gTTS
# This module is imported so that we can
# play the converted audio
import os
import pyglet
# The text that you want to convert to audio
mytext = 'Welcome to Emergys!, Your reporting manager is Python 3.7'
# Language in which you want to convert
language = 'en'
# Passing the text and language to the engine,
# here we have marked slow=False. Which tells
# the module that the converted audio should
# have a high speed
myobj = gTTS(text=mytext, lang=language, slow=False)
# Saving the converted audio in a mp3 file named
# welcome
myobj.save("welcome.mp3")
# Playing the converted file
play1 = pyglet.media.load('welcome.mp3')
play1.play()
pyglet.app.run()
play1.exit()
Example-15
# Handwritten numbers using sklearn
#
import matplotlib.pyplot as plt
from sklearn import datasets, svm, metrics
digits = datasets.load_digits()
images_and_labels = list(zip(digits.images, digits.target))
for index, (image, label) in enumerate(images_and_labels[:4]):
plt.subplot(2, 4, index + 1)
plt.axis('off')
plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
plt.title('Training: %i' % label)
n_samples = len(digits.images)
data = digits.images.reshape((n_samples, -1))
classifier = svm.SVC(gamma=0.001)
classifier.fit(data[:n_samples // 2], digits.target[:n_samples // 2])
expected = digits.target[n_samples // 2:]
predicted = classifier.predict(data[n_samples // 2:])
print("Classifier %s:\n%s\n"
% (classifier, metrics.classification_report(expected, predicted)))
print("Confusion matrix:\n%s" % metrics.confusion_matrix(expected, predicted))
images_and_predictions = list(zip(digits.images[n_samples // 2:], predicted))
for index, (image, prediction) in enumerate(images_and_predictions[:4]):
plt.subplot(2, 4, index + 5)
plt.axis('off')
plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
plt.title('Prediction: %i' % prediction)
plt.show()
Output
Classifier SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
decision_function_shape='ovr', degree=3, gamma=0.001, kernel='rbf',
max_iter=-1, probability=False, random_state=None, shrinking=True,
tol=0.001, verbose=False):
precision recall f1-score support
0 1.00 0.99 0.99 88
1 0.99 0.97 0.98 91
2 0.99 0.99 0.99 86
3 0.98 0.87 0.92 91
4 0.99 0.96 0.97 92
5 0.95 0.97 0.96 91
6 0.99 0.99 0.99 91
7 0.96 0.99 0.97 89
8 0.94 1.00 0.97 88
9 0.93 0.98 0.95 92
micro avg 0.97 0.97 0.97 899
macro avg 0.97 0.97 0.97 899
weighted avg 0.97 0.97 0.97 899
Confusion matrix:
[[87 0 0 0 1 0 0 0 0 0]
[ 0 88 1 0 0 0 0 0 1 1]
[ 0 0 85 1 0 0 0 0 0 0]
[ 0 0 0 79 0 3 0 4 5 0]
[ 0 0 0 0 88 0 0 0 0 4]
[ 0 0 0 0 0 88 1 0 0 2]
[ 0 1 0 0 0 0 90 0 0 0]
[ 0 0 0 0 0 1 0 88 0 0]
[ 0 0 0 0 0 0 0 0 88 0]
[ 0 0 0 1 0 1 0 0 0 90]]
Machine learning is a complex task. MLflow is an open source platform for managing the end-to-end machine learning .
We can use MLflow to log results to local files or to a server, then compare multiple runs. Using the web UI, we can view and compare the output of multiple runs
Installing mfflow
pip install mlflow
SQLite is a C-language library that implements a small, SQL database engine. SQLite is the most used database engine in the world. The database file format is cross-platform - you can freely copy a database between 32-bit and 64-bit systems.
import sqlite3
from sqlite3 import Error
def create_connection(db_file):
""" create a database connection to a SQLite database """
conn = None
try:
conn = sqlite3.connect(db_file)
print(sqlite3.version)
except Error as e:
print(e)
finally:
if conn:
conn.close()
if __name__ == '__main__':
create_connection(r"e:\test.db")
Vaex is a python library to visualize and explore Big tabular datas. It can calculate statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid up to a billion rows per second.
Installing vaex
pip install vaex-core vaex-viz vaex-jupyter vaex-arrow vaex-server vaex-ui vaex-hdf5 vaex-astro vaex-distributed
