Use for Loop through the value of The process It's called iteration , such as ： Use for Loop through the list to get the value
for value in [2, 3, 4]: print(value)
Standard concept ： Defined in class
__iter__Method , And the object created with this class is an iteratable object Simple memory ： Use for Loop through the value of object It's called an iteratable object , such as ： list 、 Tuples 、 Dictionaries 、 aggregate 、range、 character string
from collections import Iterable
result = isinstance((3, 5), Iterable)
print(" Whether a tuple is an iteratable object :", result)
result = isinstance([3, 5], Iterable)
print(" Whether the list is an iteratable object :", result)
result = isinstance({"name": " Zhang San "}, Iterable)
print(" Whether the dictionary is an iteratable object :", result)
result = isinstance("hello", Iterable)
print(" Whether the string is an iteratable object :", result)
result = isinstance({3, 5}, Iterable)
print(" Whether the collection is an iteratable object :", result)
result = isinstance(range(5), Iterable)
print("range Whether it is an iteratable object :", result)
result = isinstance(5, Iterable)
print(" Whether an integer is an iteratable object :", result)
result = isinstance(5, int)
print(" Is the integer int Type object :", result)
class Student(object):
pass
stu = Student()
result = isinstance(stu, Iterable)
print("stu Whether it is an iteratable object :", result)
result = isinstance(stu, Student)
print("stu Whether it is Student Object of type :", result)Implement... In a class
__iter__Method
from collections import Iterable class MyList(object): def __init__(self): self.my_list = list() def append_item(self, item): self.my_list.append(item) def __iter__(self): pass my_list = MyList() my_list.append_item(1) my_list.append_item(2) result = isinstance(my_list, Iterable) print(result) for value in my_list: print(value)
Execution results :
Traceback (most recent call last): True File "/Users/hbin/Desktop/untitled/aa.py", line 24, in <module> for value in my_list: TypeError: iter() returned non-iterator of type 'NoneType'
It can be seen from the execution results , Iterators are required to traverse iteratable objects and get data in turn
Provide a __iter__ The created object is an iteratable object , Iteratable objects need iterators to complete data iteration
Custom iterator objects : Defined in class
__iter__and__next__The object created by the method is the iterator object
from collections import Iterable
from collections import Iterator
class MyList(object):
def __init__(self):
self.my_list = list()
def append_item(self, item):
self.my_list.append(item)
def __iter__(self):
my_iterator = MyIterator(self.my_list)
return my_iterator
class MyIterator(object):
def __init__(self, my_list):
self.my_list = my_list
self.current_index = 0
result = isinstance(self, Iterator)
print("MyIterator Whether the created object is an iterator :", result)
def __iter__(self):
return self
def __next__(self):
if self.current_index < len(self.my_list):
self.current_index += 1
return self.my_list[self.current_index - 1]
else:
raise StopIteration
my_list = MyList()
my_list.append_item(1)
my_list.append_item(2)
result = isinstance(my_list, Iterable)
print(result)
for value in my_list:
print(value)Running results :
True MyIterator Whether the created object is an iterator : True 1 2
iter function : Get iterators for iteratable objects , Will call... On the iteratable object __iter__ Method
next function : Get the next value in the iterator , Will call the... On the iterator object __next__ Method
class MyList(object): def __init__(self): self.my_list = list() def append_item(self, item): self.my_list.append(item) def __iter__(self): my_iterator = MyIterator(self.my_list) return my_iterator class MyIterator(object): def __init__(self, my_list): self.my_list = my_list self.current_index = 0 def __iter__(self): return self def __next__(self): if self.current_index < len(self.my_list): self.current_index += 1 return self.my_list[self.current_index - 1] else: raise StopIteration my_list = MyList() my_list.append_item(1) my_list.append_item(2) my_iterator = iter(my_list) print(my_iterator) while True: try: value = next(my_iterator) print(value) except StopIteration as e: break
for item in Iterable The essence of a cycle is to go through it first iter() Function gets an iterable object Iterable The iterator , The obtained iterator is then called again and again next() Method to get the next value and assign it to item, When you meet StopIteration The loop ends after the exception of .
for item in Iterator Iterators for loops , Keep calling next() Method to get the next value and assign it to item, When you meet StopIteration The loop ends after the exception of .
We found that the core function of iterators is that they can be passed next() Function to return the next data value . If the data value returned each time is not read in an existing data set , But through the program according to a certain law of calculation generated , That means you can no longer rely on an existing set of data , That is, it is no longer necessary to cache all the data to be iterated at once for subsequent reading , This can save a lot of storage （ Memory ） Space .
for instance , such as , There is a famous fipolacci sequence in mathematics （Fibonacci）, The first term in the sequence is zero 0, The second term is 1, Each of the subsequent Numbers can be obtained by adding the first two Numbers together ：
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, ...
Now we want to pass for...in... Loop through the beginning of the iterative Fibonacci sequence n Number . So this Fibonacci sequence we can implement with iterators , Each iteration USES math to generate the next number .
class Fibonacci(object): def __init__(self, num): self.num = num self.a = 0 self.b = 1 self.current_index = 0 def __iter__(self): return self def __next__(self): if self.current_index < self.num: result = self.a self.a, self.b = self.b, self.a + self.b self.current_index += 1 return result else: raise StopIteration fib = Fibonacci(5) for value in fib: print(value)
Execution results :
0 1 1 2 3
The function of an iterator is to record the location of the current data in order to get the value of the next location
Generators are a special kind of iterators , It doesn't need to be written like the class above
__iter__() and __next__()The method , It's more convenient to use , It can still be used next Functions and for Value cycle
The first method is simple , Just put a list into a generative [ ] Change to ( )
my_list = [i * 2 for i in range(5)] print(my_list) my_generator = (i * 2 for i in range(5)) print(my_generator) for value in my_generator: print(value)
Execution results :
[0, 2, 4, 6, 8] <generator object <genexpr> at 0x101367048> 0 2 4 6 8
stay def You can see in the function yield Keyword is the generator
def fibonacci(num):
a = 0
b = 1
current_index = 0
print("--11---")
while current_index < num:
result = a
a, b = b, a + b
current_index += 1
print("--22---")
yield result
print("--33---")
fib = fibonacci(5)
value = next(fib)
print(value)
value = next(fib)
print(value)
value = next(fib)
print(value) In a generator implementation , We're going to be in the iterator __next__ The basic logic implemented in the method is implemented in a function , But the number that will be returned for each iteration return Instead of yield, At this point, the newly defined function is no longer a function , It is a generator 了 .
Simply speaking ： As long as def There is yield Keywords It's called generator
def fibonacci(num):
a = 0
b = 1
current_index = 0
print("--11---")
while current_index < num:
result = a
a, b = b, a + b
current_index += 1
print("--22---")
yield result
print("--33---")
return " Hee hee "
fib = fibonacci(5)
value = next(fib)
print(value)
try:
value = next(fib)
print(value)
except StopIteration as e:
print(e.value)Tips :
The generator uses return There is no problem with keyword Syntax , But the code executes to return Statement will stop the iteration , Throw the stop iteration exception .
Used yield Keyword function is no longer a function , It's the generator .（ Used yield The function of is the generator ）
Code execution to yield Will suspend , Then return the result , The next time you start the generator, it will continue to execute at the suspended position
Each time you start the generator, you return a value , Multiple starts can return multiple values , That is to say yield Multiple values can be returned
return The value can only be returned once , Code execution to return Statement stops the iteration , Throw the stop iteration exception
send Method can pass parameters when starting the generator
def gen(): i = 0 while i<5: temp = yield i print(temp) i+=1
Execution results :
In [43]: f = gen()
In [44]: next(f)
Out[44]: 0
In [45]: f.send('haha')
haha
Out[45]: 1
In [46]: next(f)
None
Out[46]: 2
In [47]: f.send('haha')
haha
Out[47]: 3
In [48]:Be careful : If you start the generator for the first time, use send Method , Then parameters can only be passed in None, Generally, the first time you start the generator, you use next function
coroutines , Also called tasklet , fibers , Also known as user level threads , Multitasking without opening up threads , That is, multitasking in the case of a single thread , Multiple tasks are executed alternately in a certain order Popular understanding as long as def There is only one yield The keyword indicates that it is a co process
Collaborative process is also a way to realize multitasking
import time
def work1():
while True:
print("----work1---")
yield
time.sleep(0.5)
def work2():
while True:
print("----work2---")
yield
time.sleep(0.5)
def main():
w1 = work1()
w2 = work2()
while True:
next(w1)
next(w2)
if __name__ == "__main__":
main()Running results ：
----work1--- ----work2--- ----work1--- ----work2--- ----work1--- ----work2--- ----work1--- ----work2--- ----work1--- ----work2--- ----work1--- ----work2--- ... Omit ...
The tasks executed between collaborative processes are executed alternately in a certain order
In order to better use the process to complete multiple tasks ,python Medium greenlet The module encapsulates it , This makes the switching task easier
Use the following command to install greenlet modular :
pip3 install greenlet
Use collaboration to accomplish multitasking
import time
import greenlet
# Mission 1
def work1():
for i in range(5):
print("work1...")
time.sleep(0.2)
# Switch to coroutine 2 It performs the corresponding tasks
g2.switch()
# Mission 2
def work2():
for i in range(5):
print("work2...")
time.sleep(0.2)
# Switch to the first cooperation process and execute the corresponding task
g1.switch()
if __name__ == '__main__':
# Create the task corresponding to the specified process
g1 = greenlet.greenlet(work1)
g2 = greenlet.greenlet(work2)
# Switch to the first cooperation process and execute the corresponding task
g1.switch()Running effect
work1... work2... work1... work2... work1... work2... work1... work2... work1... work2...
greenlet We've implemented the coroutine , But this has to be switched manually , Here is a comparison greenlet A more powerful third-party library that can automatically switch tasks , That's it gevent.
gevent Enclosed inside greenlet, The principle is when a greenlet encounter IO( refer to input output Input and output , Such as network 、 File operation, etc. ) In operation , Like access to the network , And then automatically switch to something else greenlet, wait until IO Operation is completed , Then switch back to continue execution at an appropriate time .
because IO Operation time consuming , Always keep the program waiting , With gevent Automatically switch the coroutine for us , I guarantee there will always be greenlet Running , Instead of waiting IO
pip3 install gevent
import gevent def work(n): for i in range(n): print(gevent.getcurrent(), i) g1 = gevent.spawn(work, 5) g2 = gevent.spawn(work, 5) g3 = gevent.spawn(work, 5) g1.join() g2.join() g3.join()
Running results
<Greenlet "Greenlet-0" at 0x26d8c970488: work(5)> 0 <Greenlet "Greenlet-1" at 0x26d8c970598: work(5)> 0 <Greenlet "Greenlet-2" at 0x26d8c9706a8: work(5)> 0 <Greenlet "Greenlet-0" at 0x26d8c970488: work(5)> 1 <Greenlet "Greenlet-1" at 0x26d8c970598: work(5)> 1 <Greenlet "Greenlet-2" at 0x26d8c9706a8: work(5)> 1 <Greenlet "Greenlet-0" at 0x26d8c970488: work(5)> 2 <Greenlet "Greenlet-1" at 0x26d8c970598: work(5)> 2 <Greenlet "Greenlet-2" at 0x26d8c9706a8: work(5)> 2 <Greenlet "Greenlet-0" at 0x26d8c970488: work(5)> 3 <Greenlet "Greenlet-1" at 0x26d8c970598: work(5)> 3 <Greenlet "Greenlet-2" at 0x26d8c9706a8: work(5)> 3 <Greenlet "Greenlet-0" at 0x26d8c970488: work(5)> 4 <Greenlet "Greenlet-1" at 0x26d8c970598: work(5)> 4 <Greenlet "Greenlet-2" at 0x26d8c9706a8: work(5)> 4
You can see ,3 individual greenlet It's running in turn, not alternately
import gevent def work(n): for i in range(n): print(gevent.getcurrent(), i) gevent.sleep(1) g1 = gevent.spawn(work, 5) g2 = gevent.spawn(work, 5) g3 = gevent.spawn(work, 5) g1.join() g2.join() g3.join()
Running results
<Greenlet at 0x7fa70ffa1c30: f(5)> 0 <Greenlet at 0x7fa70ffa1870: f(5)> 0 <Greenlet at 0x7fa70ffa1eb0: f(5)> 0 <Greenlet at 0x7fa70ffa1c30: f(5)> 1 <Greenlet at 0x7fa70ffa1870: f(5)> 1 <Greenlet at 0x7fa70ffa1eb0: f(5)> 1 <Greenlet at 0x7fa70ffa1c30: f(5)> 2 <Greenlet at 0x7fa70ffa1870: f(5)> 2 <Greenlet at 0x7fa70ffa1eb0: f(5)> 2 <Greenlet at 0x7fa70ffa1c30: f(5)> 3 <Greenlet at 0x7fa70ffa1870: f(5)> 3 <Greenlet at 0x7fa70ffa1eb0: f(5)> 3 <Greenlet at 0x7fa70ffa1c30: f(5)> 4 <Greenlet at 0x7fa70ffa1870: f(5)> 4 <Greenlet at 0x7fa70ffa1eb0: f(5)> 4
import gevent
import time
from gevent import monkey
monkey.patch_all()
def work1(num):
for i in range(num):
print("work1....")
time.sleep(0.2)
def work2(num):
for i in range(num):
print("work2....")
time.sleep(0.2)
if __name__ == '__main__':
g1 = gevent.spawn(work1, 3)
g2 = gevent.spawn(work2, 3)
g1.join()
g2.join()Running results
work1.... work2.... work1.... work2.... work1.... work2....
The current program is an endless loop and can also have time-consuming operations , You don't need to add join The method , Because the program needs to run all the time, it won't exit
Sample code
import gevent
import time
from gevent import monkey
monkey.patch_all()
def work1(num):
for i in range(num):
print("work1....")
time.sleep(0.2)
def work2(num):
for i in range(num):
print("work2....")
time.sleep(0.2)
if __name__ == '__main__':
g1 = gevent.spawn(work1, 3)
g2 = gevent.spawn(work2, 3)
while True:
print(" Execute... In main thread ")
time.sleep(0.5)Execution results :
Execute... In main thread work1.... work2.... work1.... work2.... work1.... work2.... Execute in main thread execute in main thread execute in main thread .. Omit ..
If too many coprocesses are used , If you want to start them, you need to use them one by one join() Method to block the main thread , In this way, the code will be too redundant , have access to gevent.joinall() Method to start the required coroutine
The sample code
import time
import gevent
def work1():
for i in range(5):
print("work1 work {}".format(i))
gevent.sleep(1)
def work2():
for i in range(5):
print("work2 work {}".format(i))
gevent.sleep(1)
if __name__ == '__main__':
w1 = gevent.spawn(work1)
w2 = gevent.spawn(work2)
gevent.joinall([w1, w2]) 1. process 、 Threads 、 Collaborative processes can accomplish multiple tasks , You can choose to use... According to your actual development needs
2. Because of the thread 、 The collaborative process requires very few resources , So threads and coroutines are most likely to be used
3. The least resources are needed to develop the cooperation process
