本文共 2942 字，大约阅读时间需要 9 分钟。
in daily work, it's common to need to migrate data between different databases. here are some simple methods to consider:
kettle's table copy wizard
previously wrote a blog post about this: a simple guide to using kettle for database migrations.use csv as intermediary
requires time to process field data types and ensure data consistency.utilize sqlalchemy
wrote a blog post about this too: a step-by-step guide to using sqlalchemy for database migrations. the process involves creating models and manually mapping field types.assuming you need to migrate the emp_master table from sql server to sqlite, follow these steps:
create the target database schema
usesqlacodegen to generate sqlalchemy models based on the source database:sqlacodegen mssql+pymssql://user:pwd@localhost:1433/testdb > models.py --tables emp_master
adjust the generated code manually to match your needs:
# models.pyfrom sqlalchemy import Column, Integer, Stringfrom sqlalchemy.ext.declarative import declarative_baseBase = declarative_base()class EmpMaster(Base): __tablename__ = 'emp_master' emp_id = Column(Integer, primary_key=True) gender = Column(String(10)) age = Column(Integer) email = Column(String(50)) phone_nr = Column(String(20)) education = Column(String(20)) marital_stat = Column(String(20)) nr_of_children = Column(Integer)
create the database and table using sqlalchemy:
# create_schema.pyfrom sqlalchemy import create_enginefrom models import Baseengine = create_engine('sqlite:///employees.db')Base.metadata.create_all(engine)migrate data using pandas
read data from source database to a pandas dataframe and write it to the target database:# data_migrate.pyfrom sqlalchemy import create_engineimport pandas as pdsource_engine = create_engine('mssql+pymssql://user:pwd@localhost:1433/testdb')target_engine = create_engine('sqlite:///employees.db')df = pd.read_sql('emp_master', source_engine)df.to_sql('emp_master', target_engine, index=False, if_exists='replace')pandas provides a convenient way to handle data transformation and export to various database formats. its read_sql() function simplifies data extraction from databases, while to_sql() handles the insertion process.
pandas is lightweight and efficient for data migration tasks. it allows for quick data visualization and manipulation before storage in the target database.
by following these steps, you can efficiently migrate your database while minimizing risks and ensuring data consistency.
转载地址：http://zjthz.baihongyu.com/