Memory pool technology of Python

Python Memory pool technology

List of articles

• • • Python Memory pool technology
    • • Small integer object
      • String resident

problem ： If objects are created and destroyed frequently , There will be a lot of memory fragments , It will eventually affect the performance of the system . And in practice , We are really doing this , Especially for the use of small integers ,

such as 1,2,3 these int Data of type , Almost every use for Loops use them . namely ：

1. Small integers are frequently used and destroyed
2. Frequent creation and destruction of objects will generate memory fragments

Small integer object

stay python Object pool technology is provided in .int Type data is an immutable object , This means that it can be shared , stay python After starting , Will apply for a piece of memory in the memory , Store frequently used small integers here , During the whole process , These small integers always exist , Will not be destroyed , Their use , Just increase their reference count .

How many integers are cached ？ This range is actually small , Only [-5, 257]. Of course, if you want to expand this range , You can choose to modify python Source code to solve .

stay python In the interactive interpreter ：

>>> a = 12
>>> b = 12
>>> id(a)
140705565998496
>>> id(b)
140705565998496
>>> a = 257
>>> b = 257
>>> id(a)
2389141537712
>>> id(b)
2389141537904

Look at the results above , It can be found in the cache , Its memory address remains unchanged , conversely , Then change .

String resident

The same is true for strings . Suppose there is 100 A variable , All assigned to python, Is it really necessary to create in memory 100 Objects ？ So ,python Provides intern Mechanism . Simply speaking ,python Maintain a dictionary internally (interned), When a string needs to reside , Just go to interned Check whether this string already exists , If it exists, increase the reference count , If it doesn't exist, it will be added to the dictionary .

Use resident Technology , There are two benefits ：

1. Save memory
2. String comparison , Resident strings compare much faster than non resident strings

When does it happen

stay python It will be more accurate to verify in the interactive interpreter

1. A dwell occurs at compile time , Runtime does not reside

   s1 = 'py' + 'thon'
   print(s1 is 'python')
   a = 'py'
   b = 'thon'
   print(a+b is 'python')
   # Output results 
   # True
   # False
   

   The reason for the difference in results ：s1 The value is calculated at the compilation stage , So it will stay , and a+b Only in the operation phase will it be calculated , Therefore, there is no resident

2. Only upper and lower case letters 、 Numbers 、 Occurs when underlining

   s1 = 'python'
   s2 = 'python'
   print(s1 is s2)
   a1 = 'pyth on'
   b2 = 'pyth on'
   print(a1 is b2)
   # Output results 
   # True
   # False
   

3. String length is 0 or 1

   Empty string and length 1 By default, all strings of will reside ,python Think that such strings are often used strings

4. By sys.intern Specify residency

   from sys import intern
   s1 = intern('python!')
   s2 = intern('python!')
   print(s1 is s2)
   

5. Multiply (*) The resulting string

   This part is more complex rules , Let's first look at the multiplier 1 The situation of ：

   1. The string contains only underscores , Numbers , Letter , Default resident

   2. String length is less than or equal to 1, Default resident

      >>> s1 = "hello"
      >>> s2 = s1*1
      >>> s1 is s2
      True
      

   If the multiplier is greater than 2, The following rules ：

   1. The string contains only underscores , Numbers , Letters and the length is less than or equal to 20, Default resident

   2. When there are other characters , Whatever the length , Do not stay

      >>> s1 = "pythonpythonpython"
      >>> s2 = "python"*3
      >>> s1 is s2
      True
      >>> s1 = "&&&"
      >>> s2 = "&" * 3
      >>> s1 is s2
      False