Full of dry goods! 20 Python tips

1. Confusing operation

This section is for some Python Compare confusing operations .

1.1 There are put back random sampling and no put back random sampling

import random
random.choices(seq, k=1) # The length is k Of list, There is a return sample
random.sample(seq, k) # The length is k Of list, No return sampling 

1.2 lambda The parameters of the function

func = lambda y: x + y # x The value of is bound when the function runs
func = lambda y, x=x: x + y # x The value of is bound when the function is defined 

1.3 copy and deepcopy

import copy
y = copy.copy(x) # Copy only the top layer
y = copy.deepcopy(x) # Copy all nested parts 

When replication and variable aliases are combined , Easy to confuse ：

a = [1, 2, [3, 4]]
# Alias.
b_alias = a
assert b_alias == a and b_alias is a
# Shallow copy.
b_shallow_copy = a[:]
assert b_shallow_copy == a and b_shallow_copy is not a and b_shallow_copy[2] is a[2]
# Deep copy.
import copy
b_deep_copy = copy.deepcopy(a)
assert b_deep_copy == a and b_deep_copy is not a and b_deep_copy[2] is not a[2]

Changes to the alias will affect the original variable ,（ shallow ） The element in the copy is the alias of the element in the original list , Deep replication is recursive , Modifications to deep replication do not affect the original variable .

1.4 == and is

x == y # Whether the two reference objects have the same value
x is y # Whether two references point to the same object 

1.5 Judgment type

type(a) == int # Ignore polymorphism in object-oriented design
isinstance(a, int) # The polymorphism in object-oriented design is considered 

1.6 String search

str.find(sub, start=None, end=None); str.rfind(...) # If we can't find a way back -1
str.index(sub, start=None, end=None); str.rindex(...) # If not found, throw ValueError abnormal 

1.7 List Backward index

It's just a matter of habit , When indexing forward, the subscript starts from 0 Start , If the reverse index also wants to start from 0 You can use ~.

print(a[-1], a[-2], a[-3])
print(a[~0], a[~1], a[~2])

2. Commonly used tools

2.1 Reading and writing CSV file

import csv
# nothing header Read and write
with open(name, 'rt', encoding='utf-8', newline='') as f: # newline='' Give Way Python Do not treat line breaks uniformly
for row in csv.reader(f):
print(row[0], row[1]) # CSV All the data I read is str type
with open(name, mode='wt') as f:
f_csv = csv.writer(f)
f_csv.writerow(['symbol', 'change'])
# Yes header Read and write
with open(name, mode='rt', newline='') as f:
for row in csv.DictReader(f):
print(row['symbol'], row['change'])
with open(name, mode='wt') as f:
header = ['symbol', 'change']
f_csv = csv.DictWriter(f, header)
f_csv.writeheader()
f_csv.writerow({'symbol': xx, 'change': xx})

Be careful , When CSV An error will be reported when the file is too large ：_csv.Error: field larger than field limit (131072), Solve by modifying the upper limit

import sys
csv.field_size_limit(sys.maxsize)

csv You can also read \t Segmented data

f = csv.reader(f, delimiter='\t')

2.2 Iterator tool

itertools There are many tools in the iterator definition , For example, subsequence tools ：

import itertools
itertools.islice(iterable, start=None, stop, step=None)
# islice('ABCDEF', 2, None) -> C, D, E, F
itertools.filterfalse(predicate, iterable) # To filter out predicate by False The elements of
# filterfalse(lambda x: x < 5, [1, 4, 6, 4, 1]) -> 6
itertools.takewhile(predicate, iterable) # When predicate by False Stop iterating
# takewhile(lambda x: x < 5, [1, 4, 6, 4, 1]) -> 1, 4
itertools.dropwhile(predicate, iterable) # When predicate by False Start iteration at
# dropwhile(lambda x: x < 5, [1, 4, 6, 4, 1]) -> 6, 4, 1
itertools.compress(iterable, selectors) # according to selectors Each element is True or False Make a selection
# compress('ABCDEF', [1, 0, 1, 0, 1, 1]) -> A, C, E, F

Sequence order ：

sorted(iterable, key=None, reverse=False)
itertools.groupby(iterable, key=None) # Group by value ,iterable Need to be sorted first
# groupby(sorted([1, 4, 6, 4, 1])) -> (1, iter1), (4, iter4), (6, iter6)
itertools.permutations(iterable, r=None) # array , The return value is Tuple
# permutations('ABCD', 2) -> AB, AC, AD, BA, BC, BD, CA, CB, CD, DA, DB, DC
itertools.combinations(iterable, r=None) # Combine , The return value is Tuple
itertools.combinations_with_replacement(...)
# combinations('ABCD', 2) -> AB, AC, AD, BC, BD, CD

Merge multiple sequences ：

itertools.chain(*iterables) # Direct splicing of multiple sequences
# chain('ABC', 'DEF') -> A, B, C, D, E, F
import heapq
heapq.merge(*iterables, key=None, reverse=False) # Multiple sequences are spliced in sequence
# merge('ABF', 'CDE') -> A, B, C, D, E, F
zip(*iterables) # Stop when the shortest sequence is exhausted , The result can only be consumed once
itertools.zip_longest(*iterables, fillvalue=None) # Stop when the longest sequence is exhausted , The result can only be consumed once 

2.3 Counter

The counter can count the number of occurrences of each element in an iteratable object .

import collections
# establish
collections.Counter(iterable)
# The frequency of
collections.Counter[key] # key Frequency of occurrence
# return n The element with the highest frequency and its corresponding frequency , If n by None, Return all elements
collections.Counter.most_common(n=None)
# Insert / to update
collections.Counter.update(iterable)
counter1 + counter2; counter1 - counter2 # counter Addition and subtraction
# Check whether the constituent elements of two strings are the same
collections.Counter(list1) == collections.Counter(list2)

2.4 With default values Dict

When access does not exist Key when ,defaultdict It is set to a default value .

import collections
collections.defaultdict(type) # When the first visit dict[key] when , Will call... Without parameters type, to dict[key] Provide an initial value 

2.5 Orderly Dict

import collections
collections.OrderedDict(items=None) # Keep the original insertion order during iteration 

3. High performance programming and debugging

3.1 Output error and warning messages

Output information to standard error

import sys
sys.stderr.write('')

Output warning messages

import warnings
warnings.warn(message, category=UserWarning)
# category The value is DeprecationWarning, SyntaxWarning, RuntimeWarning, ResourceWarning, FutureWarning

Controls the output of warning messages

$ python -W all # Output all warnings , Equivalent to setting warnings.simplefilter('always')
$ python -W ignore # Ignore all warnings , Equivalent to setting warnings.simplefilter('ignore')
$ python -W error # Convert all warnings to exceptions , Equivalent to setting warnings.simplefilter('error')

3.2 Test in code

Sometimes in order to debug , We want to add some code to the code , Usually a few print sentence , Can be written as ：

# In the code debug part
if __debug__:
pass

Once debugging is complete , By executing... On the command line -O Options , Will ignore this part of the code ：

$ python -0 main.py

3.3 Code style check

Use pylint You can do a lot of code style and syntax checking , Can find some errors before running

pylint main.py

3.4 Code takes time

Time consuming testing

$ python -m cProfile main.py

Testing a block of code takes time

# Code block time-consuming definition
from contextlib import contextmanager
from time import perf_counter
@contextmanager
def timeblock(label):
tic = perf_counter()
try:
yield
finally:
toc = perf_counter()
print('%s : %s' % (label, toc - tic))
# Code blocks take time to test
with timeblock('counting'):
pass

Some principles of code time-consuming optimization

• Focus on optimizing where performance bottlenecks occur , Not all the code .
• Avoid using global variables . The lookup of local variables is faster than that of global variables , It is usually faster to define the code of global variables in functions 15%-30%.
• Avoid using . Access properties . Use from module import name Will be faster , Member variables of classes that will be accessed frequently self.member Put it into a local variable .
• Try to use built-in data structures .str, list, set, dict Etc C Realization , It runs very fast .
• Avoid creating unnecessary intermediate variables , and copy.deepcopy().
• String splicing , for example a + ':' + b + ':' + c Will create a lot of useless intermediate variables ,':',join([a, b, c]) It will be much more efficient . In addition, we need to consider whether string splicing is necessary , for example print(':'.join([a, b, c])) Efficient than print(a, b, c, sep=':') low .

4. Python Other skills

4.1 argmin and argmax

items = [2, 1, 3, 4]
argmin = min(range(len(items)), key=items.__getitem__)

argmax Empathy .

4.2 Transpose 2D list

A = [['a11', 'a12'], ['a21', 'a22'], ['a31', 'a32']]
A_transpose = list(zip(*A)) # list of tuple
A_transpose = list(list(col) for col in zip(*A)) # list of list

4.3 Expand a one-dimensional list into a two-dimensional list

A = [1, 2, 3, 4, 5, 6]
# Preferred.
list(zip(*[iter(A)] * 2))