Many people see python, All I think of is IT A popular application in the industry .
But in Africa IT In similar industries ,python It is also popular , Because I repeat the trivial work every day , It's something that everyone will encounter more or less .
We're here every day Word,Excel,PPT, mail , Wechat takes a lot of time ！
Just need to master a little Python You can use programs to deal with these repetitive and annoying tasks , For example, use python Office automation , You can quickly sort out the data 、 To collect information ！
therefore ,python How should I use automatic office , After reading my answer, I will understand .
I'd like to introduce how I work every day 8 Hours , Evolved to work every day 10 Minutes of .
It doesn't involve too many technical details , After all, knowing is almost a point （ present ） enjoy （ Ed ） know （ so ） knowledge （ things ） The place of .
I'm not a programmer , College is not IT major .
I work in an enterprise whose main business is finished grain trading , The main job of my post is to make all kinds of data statistical reports 、 Standing book 、 Data charts and so on , It's all kinds of daily newspapers anyway , Do it every day , In addition, there are weekly and monthly reports …
Former colleagues in this position , Spend at least... Every day 3.5 To 4 An hour to toss Excel, Use a PivotChart 、vlookup、 Other types of functions to make an inclusion 3000 Daily reports of multiple data .
If the leader arranges any work temporarily , Then we have to work overtime to find what the leaders want from the vast amount of data .
1. When the nightmare begins ：
When I first took over the job , We have a huge , reach 70 many M Of Excel file , Every time the broken computer distributed by the company opens, it takes 20 minutes , It contains the detailed information of each car we purchased in recent ten years …… Then the data format inside is still chaotic .
For example , For example, we have a supplier whose full name is ：ABCD company with limited liability ,
So this Excel There will be
AB company 、
AB Co., LTD. 、
CD company with limited liability 、
CD Limited liability company 、
ACBD Co., LTD.
ABD company
Wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait, wait
On my first day I saw this Excel When
Later, the colleague who handed over told me , She used it Excel Of vlookup function , When it doesn't match , Just search by hand ……
then …… I ……
That's it , After nightmare months ……
Wait to 2017 At the beginning of year , See a word :
Turing Complete .
A Turing complete language can theoretically solve any algorithm .
|
|
V
obviously ,Python It's a Turing complete language .
|
|
V
that Python Should be able to handle everything for me in theory （ Having a baby shouldn't be ~~）
|
|
V
Why don't I try Python Go and finish the work for me ？
2. Some detours , But it's better than standing still
My first step is to start with data cleaning .（ Of course, I didn't know this was called data cleaning at that time ）
Let's start with the one we just said 70M Of Excel Derivation csv, Remove unnecessary formatting 、 style 、 Object etc. .
Then I want to unify the standard names of various data , Said just now , Because there are too many people , The same data , How to express it , Hundreds of thousands of data , Manual cross comparison is obviously unrealistic .（ Later I learned the word Cartesian product ）
I first used Excel Its own de duplication function , All supplier names have been de duplicated , The rest is about 5000 About names .
then , As a lazy man , I'm sure I won't compare it manually .
Python There's a library , be called fuzzywuzzy, Use it for fuzzy matching .
from fuzzywuzzy import fuzz
print(fuzz.ratio(" Content 1", " Content 2"))
Will output one 0——100 The number of , The larger the number, the more similar the two contents are .
So I wrote a loop , Put this 5000 In multiple projects , The matching degree exceeds 70% The above data sets have been screened out , Go again Excel Manual replacement inside （ Can't , At that time, I couldn't operate Excel）
Then I sorted out the positions of each line , In this way, I have a relatively accurate data source .
Then I thought about it , Put the data in Excel It's not a long-term plan , After all, hundreds of thousands of data , Just want to import the data into the database . I passed the computer grade II in College Access（ Of course , Although I passed the exam , But I didn't pass the exam ……）, Just want to import Access, But that Python I don't understand , Just can't read , It's stuck .
Later I thought of , In the one click end of so and so private server downloaded from the Internet before , There is one MySQL One click start mode of database , I tried, but I can use , Just put that MySQL Copy out the whole program 、 Clean up , As my database .（ I wanted to learn how to configure myself MySQL Here it is , I don't understand …… This database is what we provide now MySQL The predecessor of database , Yes, of course , What we offer now is a new version of the newly made non-toxic ship ）
And then use Navicat Import Excel, Use pymysql Library to read , such , I'm also the one who takes advantage of the database ~
Learn from good examples Python Whether it's employment or sideline, it's good to make money , But learn to Python Still have a learning plan . Finally, let's share a complete set of Python Learning materials , For those who want to learn Python Let's have a little help ！
Python The technical points in all directions are sorted out , Form a summary of knowledge points in various fields , The use of it is , You can find the corresponding learning resources according to the above knowledge points , Make sure you learn more comprehensively .（ At the end of the article ！）
reminder ： Limited space , Packaged folder , The way to get it is “ At the end of the article ”！！！
When I learn a certain foundation , When you have your own understanding , I will read some books compiled by my predecessors or handwritten notes , These notes detail their understanding of some technical points , These understandings are quite original , You can learn different ideas .
Watch the zero basics learning video , Watching video learning is the quickest and most effective way , Follow the teacher's ideas in the video , From foundation to depth , It's still easy to get started .
Optical theory is useless , Learn to knock together , Do it , Can you apply what you have learned to practice , At this time, we can make some practical cases to learn .
Check the learning results .
We learn Python Must be to find a well paid job , The following interview questions are from Ali 、 tencent 、 The latest interview materials of big Internet companies such as byte , And the leader Ali gave an authoritative answer , After brushing this set of interview materials, I believe everyone can find a satisfactory job .
Guarantee 100% free 】
