openvswitch2.16-test-2.16.0-36.el8s >  H a悐 !F^(*vufweW WBb."29rB ͝zc8@6O:ubט"]۴fIjwx?eҋ69CoM&HnŻck}ɾiݷLHJpX+.xa|hb` tڊ; Gu;> pa06a5254f506352397c305ac5cbc289da823b6b8466d7b40775399baf1308d18522305748f817f29c16229154636b0d06da711c1Oa悐 !F^(*v 0 {}OL'ESA!1n2R63ubJXʼO00%Υ sq>uMoCIQBݰngSSףW'I$wS dqZÆjੜ;W0/ߏoG5n%.I\zq7GS^,YaY&bԃiS5g<[8N^ +-CXx3 z0ȓ>^Ԍ1z[V4>p<l?\d & E (DJT&& 8& & &  B&  h& & &|&<T x @ (y8$9$:$G&H&IH&XpYt\&]&^b_deflt&ul&vwd&x&y XCopenvswitch2.16-test2.16.036.el8sOpen vSwitch testing utilitiesUtilities that are useful to diagnose performance and connectivity issues in Open vSwitch setup.aDx86-07.rdu2.centos.orgSystem Environment/Daemons daemon/database/utilitieshttp://www.openvswitch.org/linuxnoarch + eC/S&qq##//  c c+.4& < KcH VA큤A큤aaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaa84294e6095721cccb09f1cc181e25cc96c7af37a20b8f495547f9059182595653c81424946fdb37ebda6f3a765d7d0ea644b05ec1a84e62cc142a9e0200f05c75902160f370d5367848765db9abf4a323806e96d1de1536e99307e6bb3abf2fa7e83dd7151f1e638b002ad374a98899f6ee7da4a3223d78c0ae9e5c5c9bd80de1b7a478c7ecda73baaf95f59801b412b794ec3ec01d011eb8b1734526c8d4fb89bfa92c1b5f0ca681b8d5e69baf283024df72484536bb48af1c1801bf2e2dabd45a17e4c1f53b63085d8047cd04b3930130d12e274b74b5cdfc92a9723296f02339d52d221b3e7a6d14f4bbf9b4bf34972a98cad26b98e3dc2c78eac7cdfbc8f339d52d221b3e7a6d14f4bbf9b4bf34972a98cad26b98e3dc2c78eac7cdfbc8f2d2ede931f077bb4179e49c7f563c18e51f231686bb9bfebf416daba3aa81e452d2ede931f077bb4179e49c7f563c18e51f231686bb9bfebf416daba3aa81e457de05d4f897f27028c090cab346e5e657c3d879566af8a0945a4e9bfc8e2373c7de05d4f897f27028c090cab346e5e657c3d879566af8a0945a4e9bfc8e2373c38e91b0ed969545dbee50b8fc2ae461545d6edf12317a2e4214bab42e9af3a3938e91b0ed969545dbee50b8fc2ae461545d6edf12317a2e4214bab42e9af3a3978e4bc801fd057b4ef8ecdbf0a33d6f14371f84637663798e86598d7769c130978e4bc801fd057b4ef8ecdbf0a33d6f14371f84637663798e86598d7769c1309b9a54d0f0d14faa8fad34d81e509ba913c66270bf6feb644c4660481fcf78ee1b9a54d0f0d14faa8fad34d81e509ba913c66270bf6feb644c4660481fcf78ee197a06505592089a08acef246ff44cf2919a4bc2d41eac08bb9fa684ffa00d3c997a06505592089a08acef246ff44cf2919a4bc2d41eac08bb9fa684ffa00d3c970cc070bc1ff42d009031ba018fe8484b73f4c51bc97014806c264c28141715a70cc070bc1ff42d009031ba018fe8484b73f4c51bc97014806c264c28141715a6ae1fe068926a93f2519ba95e524662d1cda0e06f0ee96d685a484c31b254e35d39acf1b2b1206b9bd36e27bc88f40c2baa88059448b045980540309ddaf4b1c784ac074ef7f8e80f5791a55d218357c7115a0c79b6e90284020778857a8cba7ca2923d1e9d0f6429e9483ed8e58ec38b480fcb03d3e07907c5534a6aed1d79e96e2dc224d82b6ed87b93eff29d5ca3d400029d9eefc459ed331316522fb1ba2ad062d71cb8e0e77a7ae392ce0f846fa18dee9dab2e048f0f4cdcb77b257cca3c0e3ae6ed34f0592a10b82d672a2439cb125d84f30b8f6b16b7777a603b7ef2ad4fd41805ce7fb5fa4732ee9b50ecef6cfc8b7633c229b09e170be6a61d456ed3cdf73f26723df1a53f2589fc9cbfa93b7678766cbdb867e1e2cd7179cd90b74e085c38a1276ed25dbda2e78fd5c01396106ca9a353c8e557be1e0c6c85cc496558bc469070e4aec0763cb90b87a75989a1b7d6dced03a6046973fd9490fda375ee7da89294dfa95ff805f5823d255e4b398bacc37abec6541631a98055044596c47d4c998c7873351606a634492ef5a45cb8feceb67d49920c611d1b11595b6rootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootrootopenvswitch2.16-2.16.0-36.el8s.src.rpmopenvswitch2.16-test@@     /usr/libexec/platform-pythonpython(abi)python3-openvswitch2.16rpmlib(CompressedFileNames)rpmlib(FileDigests)rpmlib(PartialHardlinkSets)rpmlib(PayloadFilesHavePrefix)rpmlib(PayloadIsXz)tcpdump3.62.16.0-36.el8s3.0.4-14.6.0-14.0.4-14.0-15.2-14.14.3a@a@aa@a@@aKa@a@a@ababazay?@av@aqV@an@ahaf@aexaexaUaTU@aTU@aTU@aTU@aTU@aQ@aLl@aC1a8a.a.a.a.a@a@Open vSwitch CI - 2.16.0-36Open vSwitch CI - 2.16.0-35Open vSwitch CI - 2.16.0-34Open vSwitch CI - 2.16.0-33Open vSwitch CI - 2.16.0-32Open vSwitch CI - 2.16.0-31Timothy Redaelli - 2.16.0-30Timothy Redaelli - 2.16.0-29Timothy Redaelli - 2.16.0-28Ilya Maximets - 2.16.0-27Ilya Maximets - 2.16.0-26Open vSwitch CI - 2.16.0-25Open vSwitch CI - 2.16.0-24Open vSwitch CI - 2.16.0-23Open vSwitch CI - 2.16.0-22Open vSwitch CI - 2.16.0-21Open vSwitch CI - 2.16.0-20Open vSwitch CI - 2.16.0-19Open vSwitch CI - 2.16.0-18Open vSwitch CI - 2.16.0-17Open vSwitch CI - 2.16.0-16Dumitru Ceara - 2.16.0-15Dumitru Ceara - 2.16.0-14Dumitru Ceara - 2.16.0-13Dumitru Ceara - 2.16.0-12Dumitru Ceara - 2.16.0-11Open vSwitch CI - 2.16.0-10Timothy Redaelli - 2.16.0-9Open vSwitch CI - 2.16.0-8Open vSwitch CI - 2.16.0-7Ilya Maximets - 2.16.0-6Ilya Maximets - 2.16.0-5Ilya Maximets - 2.16.0-4Ilya Maximets - 2.16.0-3Open vSwitch CI - 2.16.0-2Flavio Leitner - 2.16.0-1- Merging upstream branch-2.16 [RH git: edae801e00] Commit list: ba7fffb832 dpif-netdev: Improve loading of packet data for undersized packets.- Merging upstream branch-2.16 [RH git: 6ad0375ff5] Commit list: 2595b7b3d1 Prepare for 2.16.3. 6caaae525c Set release date for 2.16.2. 443e3657d7 ofproto-dpif-xlate: Snoop ingress packets and update neigh cache if needed. 75d2ef9a60 tnl-neigh-cache: Do not refresh the entry while revalidating. 5d88836566 tnl-neigh-cache: Read/write expires atomically. fb42c99c15 dpif-netdev: Improve handling of IP/TCP in avx512 mfex.- Merging upstream branch-2.16 [RH git: 07b9bf085a] Commit list: f42c484445 compat: handle NF_REPEAT error on nf_conntrack_in.- Merging upstream branch-2.16 [RH git: 8708b55152] Commit list: 3e527f21cf flow: Consider dataofs when parsing TCP packets. b537e049ad tests/flowgen: Fix packet data endianness. 35244b4980 ofproto: Fix resource usage explosion due to removal of large number of flows. a201297639 ofproto: Fix resource usage explosion while processing bundled FLOW_MOD. cd0133402c tests/flowgen: Fix length field of 802.2 data link header. 2d65b8ffd2 ovs-lib: Backup and remove existing DB when joining cluster. ab01177637 docs/dpdk: Fix install doc. 38a2129524 ovs-save: Save igmp flows in ofp_parse syntax. dc77857ce2 faq: Update OVS/DPDK version table for OVS 2.13/2.14.- Merging upstream branch-2.16 [RH git: e90e06a818] Commit list: 1d8e0f861f ofproto-dpif-xlate: Fix check_pkt_larger incomplete translation.- Merging upstream branch-2.16 [RH git: 77a249d38b] Commit list: f8f2f7c9cb datapath-windows: Reset flow key after Ipv4 fragments are reassembled- python: Replace pyOpenSSL with ssl. [RH git: 0cd5867531] (#1988429) Currently, pyOpenSSL is half-deprecated upstream and so it's removed on some distributions (for example on CentOS Stream 9, https://issues.redhat.com/browse/CS-336), but since OVS only supports Python 3 it's possible to replace pyOpenSSL with "import ssl" included in base Python 3. Stream recv and send had to be splitted as _recv and _send, since SSLError is a subclass of socket.error and so it was not possible to except for SSLWantReadError and SSLWantWriteError in recv and send of SSLStream. TCPstream._open cannot be used in SSLStream, since Python ssl module requires the SSL socket to be created before connecting it, so SSLStream._open needs to create the socket, create SSL socket and then connect the SSL socket. Reported-by: Timothy Redaelli Reported-at: https://bugzilla.redhat.com/1988429 Signed-off-by: Timothy Redaelli Acked-by: Terry Wilson Tested-by: Terry Wilson Signed-off-by: Ilya Maximets Signed-off-by: Timothy Redaelli - python: socket-util: Split inet_open_active function and use connect_ex. [RH git: 2e704b371c] In an upcoming patch, PyOpenSSL will be replaced with Python ssl module, but in order to do an async connection with Python ssl module the ssl socket must be created when the socket is created, but before the socket is connected. So, inet_open_active function is splitted in 3 parts: - inet_create_socket_active: creates the socket and returns the family and the socket, or (error, None) if some error needs to be returned. - inet_connect_active: connect the socket and returns the errno (it returns 0 if errno is EINPROGRESS or EWOULDBLOCK). connect is replaced by connect_ex, since Python suggest to use it for asynchronous connects and it's also cleaner since inet_connect_active returns errno that connect_ex already returns, moreover due to a Python limitation connect cannot not be used with ssl module. inet_open_active function is changed in order to use the new functions inet_create_socket_active and inet_connect_active. Signed-off-by: Timothy Redaelli Acked-by: Terry Wilson Tested-by: Terry Wilson Signed-off-by: Ilya Maximets Signed-off-by: Timothy Redaelli - redhat: remove mlx4 support [RH git: 4c846afd24] (#1998122) Resolves: #1998122- ovsdb: Don't let transaction history grow larger than the database. [RH git: 93d1fa0bdf] (#2012949) commit 317b1bfd7dd315e241c158e6d4095002ff391ee3 Author: Ilya Maximets Date: Tue Sep 28 13:17:21 2021 +0200 ovsdb: Don't let transaction history grow larger than the database. If user frequently changes a lot of rows in a database, transaction history could grow way larger than the database itself. This wastes a lot of memory and also makes monitor_cond_since slower than usual monotor_cond if the transaction id is old enough, because re-construction of the changes from a history is slower than just creation of initial database snapshot. This is also the case if user deleted a lot of data, so transaction history still holds all of it while the database itself doesn't. In case of current lb-per-service model in ovn-kubernetes, each load-balancer is added to every logical switch/router. Such a transaction touches more than a half of a OVN_Northbound database. And each of these transactions is added to the transaction history. Since transaction history depth is 100, in worst case scenario, it will hold 100 copies of a database increasing memory consumption dramatically. In tests with 3000 LBs and 120 LSs, memory goes up to 3 GB, while holding at 30 MB if transaction history disabled in the code. Fixing that by keeping count of the number of ovsdb_atom's in the database and not allowing the total number of atoms in transaction history to grow larger than this value. Counting atoms is fairly cheap because we don't need to iterate over them, so it doesn't have significant performance impact. It would be ideal to measure the size of individual atoms, but that will hit the performance. Counting cells instead of atoms is not sufficient, because OVN users are adding hundreds or thousands of atoms to a single cell, so they are largely different in size. Signed-off-by: Ilya Maximets Acked-by: Han Zhou Acked-by: Dumitru Ceara Reported-at: https://bugzilla.redhat.com/2012949 Signed-off-by: Ilya Maximets - ovsdb: transaction: Incremental reassessment of weak refs. [RH git: e8a363db49] (#2005958) commit 4dbff9f0a68579241ac1a040726be3906afb8fe9 Author: Ilya Maximets Date: Sat Oct 16 03:20:23 2021 +0200 ovsdb: transaction: Incremental reassessment of weak refs. The main idea is to not store list of weak references in the source row, so they all don't need to be re-checked/updated on every modification of that source row. The point is that source row already knows UUIDs of all destination rows stored in the data, so there is no much profit in storing this information somewhere else. If needed, destination row can be looked up and reference can be looked up in the destination row. For the fast lookup, destination row now stores references in a hash map. Weak reference structure now contains the table and uuid of a source row instead of a direct pointer. This allows to replace/update the source row without breaking any weak references stored in destination rows. Structure also now contains the key-value pair of atoms that triggered creation of this reference. These atoms can be used to quickly subtract removed references from a source row. During reassessment, ovsdb now only needs to care about new added or removed atoms, and atoms that got removed due to removal of the destination rows, but these are marked for reassessment by the destination row. ovsdb_datum_subtract() is used to remove atoms that points to removed or incorrect rows, so there is no need to re-sort datum in the end. Results of an OVN load-balancer benchmark that adds 3K load-balancers to each of 120 logical switches and 120 logical routers in the OVN sandbox with clustered Northbound database and then removes them: Before: %CPU CPU Time CMD 86.8 00:16:05 ovsdb-server nb1.db 44.1 00:08:11 ovsdb-server nb2.db 43.2 00:08:00 ovsdb-server nb3.db After: %CPU CPU Time CMD 54.9 00:02:58 ovsdb-server nb1.db 33.3 00:01:48 ovsdb-server nb2.db 32.2 00:01:44 ovsdb-server nb3.db So, on a cluster leader the processing time dropped by 5.4x, on followers - by 4.5x. More load-balancers - larger the performance difference. There is a slight increase of memory usage, because new reference structure is larger, but the difference is not significant. Signed-off-by: Ilya Maximets Acked-by: Dumitru Ceara Reported-at: https://bugzilla.redhat.com/2005958 Signed-off-by: Ilya Maximets - Merging upstream branch-2.16 [RH git: f5366890c5] Commit list: c221c8e613 datapath-windows:Reset PseudoChecksum value only for TX direction offload case- Merging upstream branch-2.16 [RH git: 4682b76694] Commit list: b79f0369f2 ci: Make linux-prepare trust system installs.- Merging upstream branch-2.16 [RH git: cce913794e] Commit list: 2a4c87f300 Prepare for 2.16.2. aaa1439b8e Set release date for 2.16.1.- Merging upstream branch-2.16 [RH git: 29f01c4fdb] Commit list: 108176ab5a github: Stick to python 3.9.- Merging upstream branch-2.16 [RH git: 2546fa9646] Commit list: 5c5e34603b datapath-windows: add layers when adding the deferred actions- Merging upstream branch-2.16 [RH git: d572c95f69] Commit list: 458a4f75f3 ofproto-dpif-xlate: Fix zone set from non-frozen-metadata fields.- Merging upstream branch-2.16 [RH git: 557ca689f7] Commit list: 6d8190584a dpif-netdev: Fix use-after-free on PACKET_OUT of IP fragments. 44a66cc1d0 tunnel-push-pop.at: Mask source port in tunnel header.- Merging upstream branch-2.16 [RH git: a6c4770398] Commit list: 27a5848a33 ovs-ctl: Add missing description for --ovs-vswitchd-options and --ovsdb-server-options to usage(). 0300d0c0c2 dpdk-stub: Change the ERR log to DBG. cdd6dd821d dpif-netlink: Fix feature negotiation for older kernels. c2682c42cb dpif-netdev: Fix pmd thread comments to include SMC. 9377f4a465 python: idl: Avoid sending transactions when the DB is not synced up.- Merging upstream branch-2.16 [RH git: c1145b5236] Commit list: 0fd17fbb09 ipf: release unhandled packets from the batch- Merging upstream branch-2.16 [RH git: 5c05133179] Commit list: 3f692fba98 datapath-windows:adjust Offset when processing packet in POP_VLAN action- ovsdb-data: Deduplicate string atoms. [RH git: 24e7d1140e] (#2006839) commit 429b114c5aadee24ccfb16ad7d824f45cdcea75a Author: Ilya Maximets Date: Wed Sep 22 09:28:50 2021 +0200 ovsdb-server spends a lot of time cloning atoms for various reasons, e.g. to create a diff of two rows or to clone a row to the transaction. All atoms, except for strings, contains a simple value that could be copied in efficient way, but duplicating strings every time has a significant performance impact. Introducing a new reference-counted structure 'ovsdb_atom_string' that allows to not copy strings every time, but just increase a reference counter. This change allows to increase transaction throughput in benchmarks up to 2x for standalone databases and 3x for clustered databases, i.e. number of transactions that ovsdb-server can handle per second. It also noticeably reduces memory consumption of ovsdb-server. Next step will be to consolidate this structure with json strings, so we will not need to duplicate strings while converting database objects to json and back. Signed-off-by: Ilya Maximets Acked-by: Dumitru Ceara Acked-by: Mark D. Gray Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2006839 Signed-off-by: Dumitru Ceara - ovsdb-data: Add function to apply diff in-place. [RH git: df0e4bda98] (#2006851) commit 32b51326ef9c307b4acd0bacafb0218dd1372f3d Author: Ilya Maximets Date: Thu Sep 23 01:47:24 2021 +0200 ovsdb_datum_apply_diff() is heavily used in ovsdb transactions, but it's linear in terms of number of comparisons. And it also clones all the atoms along the way. In most cases size of a diff is much smaller than the size of the original datum, this allows to perform the same operation in-place with only O(diff->n * log2(old->n)) comparisons and O(old->n + diff->n) memory copies with memcpy. Using this function while applying diffs read from the storage gives a significant performance boost and allows to execute much more transactions per second. Signed-off-by: Ilya Maximets Acked-by: Mark D. Gray Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2006851 Signed-off-by: Dumitru Ceara - ovsdb-data: Optimize subtraction of sets. [RH git: 5bace82405] (#2005483) commit bb12b63176389e516ddfefce20dfa165f24430fb Author: Ilya Maximets Date: Thu Sep 23 01:47:23 2021 +0200 Current algorithm for ovsdb_datum_subtract looks like this: for-each atom in a: if atom in b: swap(atom, ) destroy(atom) quicksort(a) Complexity: Na * log2(Nb) + (Na - Nb) * log2(Na - Nb) Search Comparisons for quicksort It's not optimal, especially because Nb << Na in a vast majority of cases. Reversing the search phase to look up atoms from 'b' in 'a', and closing gaps from deleted elements in 'a' by plain memory copy to avoid quicksort. Resulted complexity: Nb * log2(Na) + (Na - Nb) Search Memory copies Subtraction is heavily used while executing database transactions. For example, to remove one port from a logical switch in OVN. Complexity of such operation if original logical switch had 100 ports goes down from 100 * log2(1) = 100 comparisons for search and 99 * log2(99) = 656 comparisons for quicksort ------------------------------ 756 comparisons in total to only 1 * log2(100) = 7 comparisons for search + memory copy of 99 * sizeof (union ovsdb_atom) bytes. We could use memmove to close the gaps after removing atoms, but it will lead to 2 memory copies inside the call, while we can perform only one to the temporary 'result' and swap pointers. Performance in cases, where sizes of 'a' and 'b' are comparable, should not change. Cases with Nb >> Na should not happen in practice. All in all, this change allows ovsdb-server to perform several times more transactions, that removes elements from sets, per second. Signed-off-by: Ilya Maximets Acked-by: Han Zhou Acked-by: Mark D. Gray Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2005483 Signed-off-by: Dumitru Ceara - ovsdb-data: Optimize union of sets. [RH git: e2a4c7d794] (#2005483) commit 51946d22274cd591dc061358fb507056fbd91420 Author: Ilya Maximets Date: Thu Sep 23 01:47:22 2021 +0200 Current algorithm of ovsdb_datum_union looks like this: for-each atom in b: if not bin_search(a, atom): push(a, clone(atom)) quicksort(a) So, the complexity looks like this: Nb * log2(Na) + Nb + (Na + Nb) * log2(Na + Nb) Comparisons clones Comparisons for quicksort for search ovsdb_datum_union() is heavily used in database transactions while new element is added to a set. For example, if new logical switch port is added to a logical switch in OVN. This is a very common use case where CMS adds one new port to an existing switch that already has, let's say, 100 ports. For this case ovsdb-server will have to perform: 1 * log2(100) + 1 clone + 101 * log2(101) Comparisons Comparisons for for search quicksort. ~7 1 ~707 Roughly 714 comparisons of atoms and 1 clone. Since binary search can give us position, where new atom should go (it's the 'low' index after the search completion) for free, the logic can be re-worked like this: copied = 0 for-each atom in b: desired_position = bin_search(a, atom) push(result, a[ copied : desired_position - 1 ]) copied = desired_position push(result, clone(atom)) push(result, a[ copied : Na ]) swap(a, result) Complexity of this schema: Nb * log2(Na) + Nb + Na Comparisons clones memory copy on push for search 'swap' is just a swap of a few pointers. 'push' is not a 'clone', but a simple memory copy of 'union ovsdb_atom'. In general, this schema substitutes complexity of a quicksort with complexity of a memory copy of Na atom structures, where we're not even copying strings that these atoms are pointing to. Complexity in the example above goes down from 714 comparisons to 7 comparisons and memcpy of 100 * sizeof (union ovsdb_atom) bytes. General complexity of a memory copy should always be lower than complexity of a quicksort, especially because these copies usually performed in bulk, so this new schema should work faster for any input. All in all, this change allows to execute several times more transactions per second for transactions that adds new entries to sets. Alternatively, union can be implemented as a linear merge of two sorted arrays, but this will result in O(Na) comparisons, which is more than Nb * log2(Na) in common case, since Na is usually far bigger than Nb. Linear merge will also mean per-atom memory copies instead of copying in bulk. 'replace' functionality of ovsdb_datum_union() had no users, so it just removed. But it can easily be added back if needed in the future. Signed-off-by: Ilya Maximets Acked-by: Han Zhou Acked-by: Mark D. Gray Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2005483 Signed-off-by: Dumitru Ceara - ovsdb: transaction: Use diffs for strong reference counting. [RH git: 85da133eaa] (#2003203) commit b2712d026eae2d9a5150c2805310eaf506e1f162 Author: Ilya Maximets Date: Tue Sep 14 00:19:57 2021 +0200 Currently, even if one reference added to the set of strong references or removed from it, ovsdb-server will walk through the whole set and re-count references to other rows. These referenced rows will also be added to the transaction in order to re-count their references. For example, every time Logical Switch Port added to a Logical Switch, OVN Northbound database server will walk through all ports of this Logical Switch, clone their rows, and re-count references. This is not very efficient. Instead, it can only increase reference counters for added references and reduce for removed ones. In many cases this will be only one row affected in the Logical_Switch_Port table. Introducing new function that generates a diff of two datum objects, but stores added and removed atoms separately, so they can be used to increase or decrease row reference counters accordingly. This change allows to perform several times more transactions that adds or removes strong references to/from sets per second, because ovsdb-server no longer clones and re-counts rows that are irrelevant to current transaction. Acked-by: Dumitru Ceara Signed-off-by: Ilya Maximets Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=2003203 Signed-off-by: Dumitru Ceara - Merging upstream branch-2.16 [RH git: 2114714012] Commit list: 547371ecdb cirrus: Reduce memory requirements for FreeBSD VMs.- redhat: use hugetlbfs group for /var/log/openvswitch when dpdk is enabled [RH git: 4e5928b671] (#2004543) Resolves: #2004543- Merging upstream branch-2.16 [RH git: 7332b410fc] Commit list: facaf5bc71 netdev-linux: Fix a null pointer dereference in netdev_linux_notify_sock(). 6e203d4873 pcap-file: Fix memory leak in ovs_pcap_open(). f50da0b267 odp-util: Fix a null pointer dereference in odp_flow_format(). 7da752e43f odp-util: Fix a null pointer dereference in odp_nsh_key_from_attr__(). bc22b01459 netdev-dpdk: Fix RSS configuration for virtio. 81706c5d43 ipf: Fix only nat the first fragment in the reass process.- Merging upstream branch-2.16 [RH git: e71f31dfd6] Commit list: 242c280f0e dpif-netdev: Fix crash when PACKET_OUT is metered.- ovsdb: monitor: Store serialized json in a json cache. [RH git: bc20330c85] (#1996152) commit 43e66fc27659af2a5c976bdd27fe747b442b5554 Author: Ilya Maximets Date: Tue Aug 24 21:00:39 2021 +0200 Same json from a json cache is typically sent to all the clients, e.g., in case of OVN deployment with ovn-monitor-all=true. There could be hundreds or thousands connected clients and ovsdb will serialize the same json object for each of them before sending. Serializing it once before storing into json cache to speed up processing. This change allows to save a lot of CPU cycles and a bit of memory since we need to store in memory only a string and not the full json object. Testing with ovn-heater on 120 nodes using density-heavy scenario shows reduction of the total CPU time used by Southbound DB processes from 256 minutes to 147. Duration of unreasonably long poll intervals also reduced dramatically from 7 to 2 seconds: Count Min Max Median Mean 95 percentile ------------------------------------------------------------- Before 1934 1012 7480 4302.5 4875.3 7034.3 After 1909 1004 2730 1453.0 1532.5 2053.6 Acked-by: Dumitru Ceara Acked-by: Han Zhou Signed-off-by: Ilya Maximets Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1996152 Signed-off-by: Ilya Maximets - raft: Don't keep full json objects in memory if no longer needed. [RH git: 4606423e8b] (#1990058) commit 0de882954032aa37dc943bafd72c33324aa0c95a Author: Ilya Maximets Date: Tue Aug 24 21:00:38 2021 +0200 raft: Don't keep full json objects in memory if no longer needed. Raft log entries (and raft database snapshot) contains json objects of the data. Follower receives append requests with data that gets parsed and added to the raft log. Leader receives execution requests, parses data out of them and adds to the log. In both cases, later ovsdb-server reads the log with ovsdb_storage_read(), constructs transaction and updates the database. On followers these json objects in common case are never used again. Leader may use them to send append requests or snapshot installation requests to followers. However, all these operations (except for ovsdb_storage_read()) are just serializing the json in order to send it over the network. Json objects are significantly larger than their serialized string representation. For example, the snapshot of the database from one of the ovn-heater scale tests takes 270 MB as a string, but 1.6 GB as a json object from the total 3.8 GB consumed by ovsdb-server process. ovsdb_storage_read() for a given raft entry happens only once in a lifetime, so after this call, we can serialize the json object, store the string representation and free the actual json object that ovsdb will never need again. This can save a lot of memory and can also save serialization time, because each raft entry for append requests and snapshot installation requests serialized only once instead of doing that every time such request needs to be sent. JSON_SERIALIZED_OBJECT can be used in order to seamlessly integrate pre-serialized data into raft_header and similar json objects. One major special case is creation of a database snapshot. Snapshot installation request received over the network will be parsed and read by ovsdb-server just like any other raft log entry. However, snapshots created locally with raft_store_snapshot() will never be read back, because they reflect the current state of the database, hence already applied. For this case we can free the json object right after writing snapshot on disk. Tests performed with ovn-heater on 60 node density-light scenario, where on-disk database goes up to 97 MB, shows average memory consumption of ovsdb-server Southbound DB processes decreased by 58% (from 602 MB to 256 MB per process) and peak memory consumption decreased by 40% (from 1288 MB to 771 MB). Test with 120 nodes on density-heavy scenario with 270 MB on-disk database shows 1.5 GB memory consumption decrease as expected. Also, total CPU time consumed by the Southbound DB process reduced from 296 to 256 minutes. Number of unreasonably long poll intervals reduced from 2896 down to 1934. Deserialization is also implemented just in case. I didn't see this function being invoked in practice. Acked-by: Dumitru Ceara Acked-by: Han Zhou Signed-off-by: Ilya Maximets Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1990058 Signed-off-by: Ilya Maximets - json: Add support for partially serialized json objects. [RH git: 885e5ce1b5] (#1990058) commit b0bca6f27aae845c3ca8b48d66a7dbd3d978162a Author: Ilya Maximets Date: Tue Aug 24 21:00:37 2021 +0200 json: Add support for partially serialized json objects. Introducing a new json type JSON_SERIALIZED_OBJECT. It's not an actual type that can be seen in a json message on a wire, but internal type that is intended to hold a serialized version of some other json object. For this reason it's defined after the JSON_N_TYPES to not confuse parsers and other parts of the code that relies on compliance with RFC 4627. With this JSON type internal users may construct large JSON objects, parts of which are already serialized. This way, while serializing the larger object, data from JSON_SERIALIZED_OBJECT can be added directly to the result, without additional processing. This will be used by next commits to add pre-serialized JSON data to the raft_header structure, that can be converted to a JSON before writing the file transaction on disk or sending to other servers. Same technique can also be used to pre-serialize json_cache for ovsdb monitors, this should allow to not perform serialization for every client and will save some more memory. Since serialized JSON is just a string, reusing the 'json->string' pointer for it. Acked-by: Dumitru Ceara Acked-by: Han Zhou Signed-off-by: Ilya Maximets Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1990058 Signed-off-by: Ilya Maximets - json: Optimize string serialization. [RH git: bb1654da63] (#1990069) commit 748010ff304b7cd2c43f4eb98a554433f0df07f9 Author: Ilya Maximets Date: Tue Aug 24 23:07:22 2021 +0200 json: Optimize string serialization. Current string serialization code puts all characters one by one. This is slow because dynamic string needs to perform length checks on every ds_put_char() and it's also doesn't allow compiler to use better memory copy operations, i.e. doesn't allow copying few bytes at once. Special symbols are rare in a typical database. Quotes are frequent, but not too frequent. In databases created by ovn-kubernetes, for example, usually there are at least 10 to 50 chars between quotes. So, it's better to count characters that doesn't require escaping and use fast data copy for the whole sequential block. Testing with a synthetic benchmark (included) on my laptop shows following performance improvement: Size Q S Before After Diff ----------------------------------------------------- 100000 0 0 : 0.227 ms 0.142 ms -37.4 % 100000 2 1 : 0.277 ms 0.186 ms -32.8 % 100000 10 1 : 0.361 ms 0.309 ms -14.4 % 10000000 0 0 : 22.720 ms 12.160 ms -46.4 % 10000000 2 1 : 27.470 ms 19.300 ms -29.7 % 10000000 10 1 : 37.950 ms 31.250 ms -17.6 % 100000000 0 0 : 239.600 ms 126.700 ms -47.1 % 100000000 2 1 : 292.400 ms 188.600 ms -35.4 % 100000000 10 1 : 387.700 ms 321.200 ms -17.1 % Here Q - probability (%) for a character to be a '\"' and S - probability (%) to be a special character ( < 32). Testing with a closer to real world scenario shows overall decrease of the time needed for database compaction by ~5-10 %. And this change also decreases CPU consumption in general, because string serialization is used in many different places including ovsdb monitors and raft. Signed-off-by: Ilya Maximets Acked-by: Numan Siddique Acked-by: Dumitru Ceara Reported-at: https://bugzilla.redhat.com/show_bug.cgi?id=1990069 Signed-off-by: Ilya Maximets - Merging upstream branch-2.16 [RH git: 7d7567e339] Commit list: 0991ea8d19 Prepare for 2.16.1.- redhat: First 2.16.0 release. [RH git: 0a1c4276cc]  !"#$%&2.16.0-36.el8sovs-l3pingovs-pcapovs-tcpdumpovs-tcpundumpovs-testovs-vlan-testovstest__init__.py__pycache____init__.cpython-36.opt-1.pyc__init__.cpython-36.pycargs.cpython-36.opt-1.pycargs.cpython-36.pycrpcserver.cpython-36.opt-1.pycrpcserver.cpython-36.pyctcp.cpython-36.opt-1.pyctcp.cpython-36.pyctests.cpython-36.opt-1.pyctests.cpython-36.pycudp.cpython-36.opt-1.pycudp.cpython-36.pycutil.cpython-36.opt-1.pycutil.cpython-36.pycvswitch.cpython-36.opt-1.pycvswitch.cpython-36.pycargs.pyrpcserver.pytcp.pytests.pyudp.pyutil.pyvswitch.pyovs-pcap.1.gzovs-tcpundump.1.gzovs-l3ping.8.gzovs-tcpdump.8.gzovs-test.8.gzovs-vlan-test.8.gz/usr/bin//usr/lib/python3.6/site-packages//usr/lib/python3.6/site-packages/ovstest//usr/lib/python3.6/site-packages/ovstest/__pycache__//usr/share/man/man1//usr/share/man/man8/-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protectioncpioxz2x86_64-redhat-linux-gnuPython script, ASCII text executabledirectoryASCII textpython 3.6 byte-compiledHTML document, ASCII text (gzip compressed data, max compression, from Unix)troff or preprocessor input, ASCII text (gzip compressed data, max compression, from Unix)troff or preprocessor input, UTF-8 Unicode text (gzip compressed data, max compression, from Unix) RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRutf-83db16a3bad63c4ba9455f61961697fe84f14cdfb372eaf815488e801dd24fb39?@7zXZ !#,V3] b2u y-iSqgmxȋ4mTh@1YNBgAƅF&W̑/(cܖn]uFADލL׮oBkg2夬|"_Wͣ>p鮴yE*:@pwi%p"A! ZqK4/pt(z~>)pxFtRs߆1r෴yo ͜ٯ^Kn%O ++C,RX|"Zxhuuvz/:X 64F(M(]y+@幧̜n4U5hr^|1erO|aаdV h'Jv ts /֐9ڴNݙM~~\ZCm3AYzoUcʃ؟s>I ~OUVGc]$nek)m6s/e+)o.5,XBI`Ұ qKS[Ѵ#;_>QMvrmV ѹ/Vv+Ģfc[qE+&a'Ncg άsoo! .4`۴3wx b2yUd' gOsfLxUԌ{R߸4  =֌i M"|uI젺)ZnUⱾ\W9CDwG ba]YG_ɤ+ r[MpB2q*ު;>])ǟi(Jq eM %,hlY>g+j#nHjA7zΤSU+R v]=S7^e6{\J]fn 6WQ.:RR֟z&$oҬgJ UY1|}Ti 񫭕|i#o4ь([ JIrUAM3^,:BNX8F]`M@v|mXQhSJy}mknY*ݫF%:¤-Z9 .5FɤO6Z; 9Fhsg'o1 nkLzpԹ}i-Ý\Ѕ>WKlȿ,ϋ;)( 1@?O 'hn/l~+A/Sgkv8["o_ScW; nwC8ο,XoBPF@0}rpByaҋ׀ 0z"X9r=gV6x M̀QW淅{( _<$Nf&X/D:T5^j+(b{椶 IY?[)i\hOz|>5 M6+f2311~$c\!DTL) "#$fL E{<$Xf%mq򛲑R ;* s_ ~*1Q Mj"؁Aj/< | "&{bLB8!A'u4S9!ZmZ<;)ѻ6,dl}01ۙC-aVe/#& b'"h0>=@WIeI6Ao(%H&rE%%_@+=D&D9qU%y9d|*wO޻$;QK3{)>t3`$$¯BF_L5#R) ʿ:ιgwtc]D1 d3z<4΃eY-cOECyzVlLgMc?-~֒I)~IFN)֍3\NRʻkn|D[iV~ۤO[;+rf^V3*fVIm"I57D怕GksTC_sۘ SsJ]߳ONjzoрxAP z)g7; D.vjthRuINtNt]P% U@粠L< : &{4n|]4^k)0 m"3~DXmoo?M~?mwf)N_},ekmTHr'ſ,OBۄ9:Ns!WEzh$ZӢLJ0{rmϮ;?g[CUzcYjD?amiZmG(!ӗmHdEP7LGT /n}.qrA+L !A2, ԾSEgw-ըa2њs5w7Ftx@ |(s0LVN?* eOyphpd׊cm^_sLx!g HY|h6r.i9ަ7JЗU30L[9<&os D0]XYϠ阮!1**d)0z+o:5z*3֫oR.%n= QeK񒘉]hKuZn_ 3ىrELt }˝V'#3b<./i8IG@b{ADěx{P0@52Pg|)(&R;^]YVB  `5Z,W_dG!YY*h&W(>)U!I=i6p6T }Rzib;Oi_a@r 1elɨAwA͒~x7YQvtj9+M{:D[}55ʖUa`2,a̚Ic 7ߠr8a&9mWZZYxixD㶾L8 |.~S钃Wevcсd^5}oD * 'ݤYp?DQ@ dh0\;ϕN)ON:zY)jThc} 㗹uxY> +&ܧzLV^@D)ڐ#: T;] aJAsTll\&d,nѷx{jv̲:ֲP lz3 ^Fqs?N\,W;*ȀOBrNOab|d{A7L/>r |l׏*>[+܉ +Ig 6Ӹ*%uAinUë_ #6UŖe&@XYTbh=Ϙ8˫LȚ? MV<ۂP^&tlމ22Du%Jk* ̥Z,$?~Ҹ\<*m g^lD wa zl.+1՝hӸkN"s}?wx2{fhf fz(P YC*춀œa]rrBل`[t"qB)U@{BN&_0#(v(v&VPO2Q)e#6W`^g9 PDLy¿!v,Z_z8 n3xFH+HCزj؍ ^\玛ԍuĉyJHDaY^ğ`C U m1 uc?r Jy|Kp:MBa44)pO*\I1\')ս>&:RƪME+Kz( \rGUֱwTJ'~e]##+VbzJeI:|ʪ8O4qۆMoHcfhQ;K -6p~,J; I*hQۄV U98jWw1};E"V ʻA?i揊hfy X2$9JioH,Ӟ?ǔ}ܤS#S Y都1VAhэmijbi'1s)/syVWcՅF3H`_hɊn|I#l-ˡ'k{dgg@b-iT5i'#'AݶXu/HYLAW3=A$\`pk#(i9NVlcDGIĐ& k DB|o+_Ծ۠)=yQ* &e* }A {jI_  2QlxO& `P?~$n:Sihiq'H=306g`{] QQTʩlq(>|pTE;jZ)ӵMQcˈ>V؟ѿݎ4gG`U!ZDv{n;EIUY)Bn]wz +zњ5:Zfߣ.ʍ}f*݁)؏#X,Lߧ϶Y ˢTyB'Sjcx#7|u L~ZKa_} c!$kk6a Ӡ kW٥qP3'O:Z~@X+B6!+j[uߓ+z( bG$! |4A/(dDn E'>^ڄg_z_h@o|ӮB=萯Oש* xҒ5XqU Fcp}c|%}")N Pg=ϛ«H&jU B^ؚUJˇMYꊷ)/yB),'O1S &E>KAt{8*]LZK"Ox1l Y>YI`=r?ˡff|PuPIs~wSw?j(7ym~Y6T yNt"Y-/0^W4]^S89󙪧 ]Tj%#jOA j.W Ҭ*\#os,"3(\㏻Y`^ Iq$%DOCخ`䊎rJaծA:>L|Mu]:qm|+Vi܄~V>f:Ϗ;ث+x3vуoZ8ϸ$6&k*F^3KΞфq_.P ȓQ#`hfH#U-Y3. )Q$Vqg9INF0_78s-IQ4FSdNdL-7EwPhÚl C8{Lpoz@,Pn>L|='~.N5@Lzbthر! +\78ǓbkM)̉寽@/aWJ{KMS,<}4gܰT5Ծ@Ar֐RԴHX|%DB|&CV/Y 0)9 S'9 F5`##%4܆M-n+\)>cKwiRqVqU/h;A@K4뮁rJ1t.HN#iUbwo-t~@WYQnsqMB3MX;X'1(A s5N$)R.GU T_BTΙ>iMe?W?HLa7)ՍΣjy0^cA.~˃1naUR@уz7&O7)c"p hQvmL]9>O)ð/b+PŽ\vYj܋Hw5@܌K 7'^حE?%w\c2P0]:fGi7a4 7s$NjVcj~6@h$՝9=Uuvix_3z6u OnH=>`s_" `>g Ԥ8ޡ^EB6Ji. }Bƻ|gi PjpN&\V\4TS(:/\:JϺ;8@7Р{2z,5ß띸꛹ykO/~=Y젎FU1g#3jH Q@&Mr٨ Z 5Sdg4oz]m3J/TN↍dKׅc dmsG.<{J)}W+FÿlJ0ӵ^œAK݆UA Jv`+VrN<Ɗa imr@o鸓"d܎ ;Mn%DkƇH|+Xr8jLqhT0쿋kN.]2ܘ̅!Wc &WzQ}ޅݚ5SͯIՓMn+gPKHa% KTD \ `rc4 J_ Rg8gY1RVMw|7buEU v+WBQ(wKTn浔cseE #Wc,E&VgE50L=] /@:pm& D$1,;Fx \u\g+akۇM]/|*&or 祖'y Ҙj 7#ZwY4 wvtm6%p,V4ѽRrpYHQ&mOu\Ĝ֌b 7e\ US2 zYg1\;?1e@Y? VCAPƺa)Mt6n^dQM IZj/K{|Z+d#G Er>`*{e6A)S$*[F;N3P8RC)qL3Kj=6\p̹02ꛊN w{@F~zӽB?=Βu U:H>F ."\ Qک7;Q{]p7AA.'$ytڍJL<\Mǔx7vĵJ414N>q_;2לm ;\:L5aױ4enQ;qK*>^xj  ΑaBe'! +7pH\iyq9E!^Mv&_F"Z7yݡұfohmEԉΞ{ZTz$րΓ`_5hµY^^kȾSe 7!5;,i|\]ZS媴6`.%|VɊ̖5,٩p_pIө]5 ,>rѤ2㢘[!}.0f^ 0f(#4Bc&R)q00#3'/KV ȡ!5Na>!ep D4« @-vX9w3h VJJ塀"=k1Ejom{%ﺻ1S\IHaΑS*7zHռyaح;\:F#$*^oMLHy<Y肅Q!IPGfLT yɹ'^wC4]vZgᩱ?ǜ*W,2:$Vf]MYD"#kaZP y9 )%k%)(y{gׅZFA~%MyiEݪ<.VMcbY_HMjNt"c~>`y!udWGht$<b#2o(D"cC=\;4<*/. .K75{ΒȑQ5, LWTyX}YCǻfhz'3T֞CF ˸oB܀٧v vP-_iE0 ?~;mN :32K5f+X?Je"~,{xJ- ?r8hƓ4>?`^ec?QjP\3*sFf8x&)lC;yYw)mc-kwʪNdV B3mޠ&W:2BWJAy~R p-ڟt{@ֈq}ϧgn3h  i|U0z)J2'#Cj]G9dj<N@?+NzxאJIn#~ߨ{ ^*!PwC$EPj.D2уJU{m,LXF~|Ju"NRˀ[;AW7=]zF{Ϯyl G y*583y_c(K@|&^Ezޛ< ]_x'WxfG 1}#0KVuSr[p};$ J|GWPpWz{?oֵ>lLQ:)xzAb|҅is: (zr-C ON_v($#rI(9g40zNe$+ZKrrV@{6Q/b6(- ܨҖ#1nAn sO<<=/S3@h W=%W6 up 5V j.hGrդu !"_-V]ĩ80+,^^]ϔF~6rd^,Pq|-2 Fa2%K¸9C#uepHEےXOcLX00wo*;P\;:dcҢT!|bLDĠ&*WDoF?.kS*ߌXѵ+9hOsQ.ʢ#,Oj)U [e?f%Ibn/\^x7l##RՁ(gYIˬA t Kus#֡Dd{2+t oߕU+e =6Q^ia/?F 阠'$IO ^-K ?!&>6^ !ejYt+Dž# Ľd3=u!ذjUu^U3pL"x_О l. ϧ **'dgyb#}`3Rhθ1 H pZfxj(tk cF ?@'[GEEy y{;$?*aֵ/WI^x)oc TS1sP[@X{)> FiAn6S [xx I:=VѐNFγhz(;^}!UtV9493s⇘Q,nxLBŢ4-O$LMH!EPֳ>rVxȔ+ajx`;6Ӥ|r&[ UƼ*0N\2.YKS D(Zק#j}#`OkrkBMGu=`- қkĨwL\?JG^,Ғzjăvf/:\ZB6.bsc^jBYsUeJD(Q߂3"Z`r .-ZgxӐ!VI.LGS|!3?6%1^ K:@x8҄zIlCI36(K}8E𷞜eӉIԩ2Ԉ0zGr>ɉtZ!y7p˂=5H>w\N9C:w&eTd *Y W!D$M]5 pNDG\քWOx"Uh}p"$7s;XɩSf(C } )^10Q1֟woKBsQD^)Dt{iÐX C0,l13a{!KӬZ"Ov g ?&^"=j A;:{c`DF{*޹9zxvJ fh}pL6)wЕǔKn"gC8QB9"n~)5OHAPQiY5̷|Zm2 v/l2Gͣ]k Ut~zr0(WO1NM^/,]S:cY-bv,g>Y'RE4:>̵4RnSm;N8+e{6[yz&e-ǽ ߝ۱鄞U79$qQk(^SȷEP E!^P7E;KͦhQ2]x]p }5O32 F:V c t: U62|͍83`$z_5V08/H,5TfOwD8&Yl3 VKk9ϗea&'c^P]@`>})1!׻9?l }/y9U,]d$\f$k-;:~]$]@iiRMWV@|0sZW"V" oMb FٷOz+Ř3]`2/#qA= $Qe8V U!1S!]lcE]T#ZG3nZ64 +wľX'p@e e *y&kS֧{N񺦜xIYܻ+ [=V P;!phM+f_,W-@ 1ЗX& }#$w>l9.ID!rPN\s$k8k1VGmtαa^KQ96@n+gvDEOx igնԼ%ZOV.)Oy_ 4v9=5H*Fsfoy*ETK'-yFN`-*f͵RNȳ,7F+643IgIvfUTZRQWM }M zB^}#`㖙DC((*0T BhSab'LX㽠DgHOˑRAR=@2]cg3zx${L4j݃PUI3F[5^!c Hi#{M::u4''{R5@-MU ֲ*wYiy"o 5 2Vɺt=9|*kEAaU! 9{qyK ^=n#:K|cOu"{aTR\ pH/}!K13پTf/U,nHO!7/"'h e/o#Ie+=d:Y[^k(ڦǓ]([ʏz&m?dEŐb)kd 5zcBE\tF|4֞AM w 8(uFRp~Nɑ `y^:km&\l ()YCZ%cJPK?N+b 0/eFQե:IQoY7!d1|]{ua,BwR-#↍d(}q: pn>ڜC#cFA 2qEx^%bب"1S@(TXEʖ2"m_ rǭYݴO/.,eaK8 ߹T3N?{9HsgV:r;.'xF~cHԦ'~LNЍy7+:/;)s?GU3{9ve7C'#"4jAwx|Lx̯e^$i%Mo܌ZXUn@BJnE_s\dO9(c?2S?QZFLē9Y/otfqEf{XLPO6. H#H @Hp봥xz?7\7$k$م4(oE7l%4ߟɃ8: y}*9}VqQ>u&Mؔ?fCJPqC)% d&b.5wJ\OXMIjכl/oVFТ_yH.A&Bw`<֚> KݦF7@#r|BRv顏w$E߁wyWlYd bp,UȜ[=~,EjjbPeM!2ϑj;dQ!.L5ZJ!Kd}, ̾%XuBS86DxxFI; 6U`1?8N07ݲKY}XNЀve{G`OsU3sov8ш }zsnq'Cv0Yz5ZMH2㇊vT491XCҮ&uZ.If?d "ܽyoIA@>{  \<_ i*m2UnYd6[uETιӚu庿=YtC3PҨxbVV-`~B:/xVˁ_ʻöQ"7iwh_f[·wǮMb#Dgkk5#(,x@fѵRGt|oؿs>]O&MeX=7"]e|1txyn fys>,xOcbwN\!EFbvGBnJ-݊Ԃ? {LIW5 zs!bt\݅W+/Ktlf|bx>#]U{qiWcgzn-S8i ,-Zg \6`Aj]}{(0U"E9)xuCZ23r.€ޞzedÈsq O 3yɔ<-H&%ʎZ}jx#ÓeQMRo~Rft;a%F&g -wt(s|ld`.%*/a'8cB]*b.-BW#;aX$618R[>wfF#?䖧RR#{GMHY>6qbU8ԍAU]B6$ڕєZXRS~<9E1fo@Kݨ!Pq, lCS>{ \U:zq]dKZmFbܹL:M ֫}r[h 4c/j ;,pZ t 8AE2yu\hx2np WTAw`΄ѿDody^Җ)r~Y=uX;~6Ϊ8#Ukч|* I,; ǧ4a գFL1;7gJFtҤp{ G05z>(&ЇN.kJ-uNʣL 8N3~*,ORYL胏\tq}\̇G5x٬tf,.mImV w`2 0'?uzOȩȺ{wl-:=^@j&}:eUy~0"al" SՄAl/?;_KQegI5Bb^MR͞bx 8jO 6 ^=:97GBW&]MJ;9ᷠB,jrw(1s:P8 ٌӋJC܍LWKYVt`9>{f |>u6bP!DNc"P߷Q!]Esl?Y|`=j.Ym #\TwDe\fŦ$Q{٤5m/UIR1^*3;4g`^%gsN:PUߟAOo^ӄbni%䥕MH"ZN49F3S?׫<7 KV v( Kkr['H0d1Sj;WAboWG2Ԥ*+ 7BW{D!_Skw@CxZ m_#^u*heF^b['%bZ j3+OZ'pS g=fxh6 P>8I<\3ߢ6G"“t4P/kyR,ȇm`vE =8]*H<8 R;aMP7!Hz'JC ];*>DiWDeh9At_eEU(LF.0q<;a<)rD j=4j~ֻG!d+`O-4KvA$u8>cLXetPeWc>X|S႗WÚyJSUwx!CNeu-Xg~9+//g"aWgajj䢨sV"v2y6\酪E#nn?xw]KI l;}^FXȖ5s_ d圙iij 49{{C4m D _uaJM!>f޼@hkr9(y2RˆgObK3 ;q3s__ްc&lyX3-5:&Vg]:X*E`x4AB]}P9,y(R  CBlXWuxxE$w0J zAMkErg4/\5oSq*EۯF{zf{񡡹 NY8[gP4ޓ\VpJ!Y^s^ vP30e 5!lO"1X  Y'*=x(칒u2L&wHQFŲ8In)+Oҟ׸Ҍu?җm&@|k,6#ӄ/]\0{>ۀ;݅:9_'Q)TJtEtO&Z PK%$  eVZpx&I `βgK.?:ϢOidM[XؕW xqS*NZW)PL9)9T2->Ze뷲dfr@H=}3jY5 e*æNįFtKxFf+hKL).+(٦jTF>1_NY)?23IQ܋oe7b"{{O`$LVu'epᐖo"K-dȴeLہ.+k eVHTyl(gY- yfz!ѕR)Lbt=ڤ |H.^-EZ D q IW洙5eEi*S@C H1b- f^H\W|V"jLR1g}ب:4㚣z l~X7a]9G'٩U-N־Uq@`۷(E!L7mMN|5uVnz[%<NJd&]1:!8!@&m9 DjZgOD':#/|e~U7cĕ- FA 9 B\-#Mhg Ox2&qcJ֯UQ!g%rqQ4LMDJv5K U}%rxib(>N)B70 esl`YyX /r$J16⋕Te)fAxY4S 9mUhr?}s7;v46)-.tͧ[e%_xBadI8ϿB/V? x&;Ҍ'h-N1l{OL. mc,i뾧#p'{96X~ʅJY Cu$dKN:G0^N[x*:4Upӵ]uU[B߇R'ㄹ$nn^8쉛IٶLu\{tgdT.PsP˨kR1+M cN{\&&phGjeA p6[hWId"("# :Pv QLZeXªCW*v g A2u[ѥ; ۻؒ;+S a gޙ0o&cDW%s7 3!CSхfsg4olaڴ6\ӅkK$ѻD4޲2%ۺ!ED)WxBİ ^wrnژϚߧ`R)P N;-b3g9'牚"UmG.kf~׃FHL:TuQ*Au j0 *;vϘCyHӢ^8<$]߷pCᤏFk}VCcϼO'*sfoGy-[CP aeJ$.W> Qx2%1~"檗AIz#9[}ؑ-Ԧ~/З='+CxȋR:QTb C\=`o*I'4_ҀJYtf |SDnԈ:Tp X 4oփ77hرLWc8oPڏ܉ sS 8rlELu 3%y|E75ɢ<;9ފ-u̒ tZM 0;XOSܢKmJt^?JF#Rjnį' ߦq/eAP8n7zSne-1$cNDQדhD~`AW*]Q Mm~47[ e ‰ìa H :~&#gCi.C (·֤޲W4M ElM}מHQɍ皕aτ/R(mr6s#3s۵>68 `L6H<ȱ7zn ELHR{M;{]@`P3|1erJ3UImaK–#CU$[LyBPWHΜ=,"RCbTڡS ],}V;Vfr=NZkʨFᝁ3zxsf Fo ~R1s#ՙ)"v9-ֽ`^нXx8 !`ihq1ж%[6e&o1rtvMAGzr's X`.h@޴RvI6FS]$ o?#SN }}y5U\qÒlѭ Y~aF4 G kg> 3DRL/x Թ3a"Vw:LfZl7[C_ivm#8^#SknXʖATQm B%Yn٪7+BUZETtnޮ7Ÿk˥b6OePՁ 4+pDimUoN{.Ǥ}}h%ۮ )bRD,]L, o"$4mz΋1\*b 8 _hF}h0*3iM&ekK˲BPH>)8|0g2T}1|:Ÿ->qs~;u8wWQ5 @T~M2q8fU67-Fi2& 1Jǚ <; WyoQbLK!A6>%hXfsr s18؄F,UB&{ >t?`Z a_cu 馅f€ӑUJѹ-R*pj@0T\8rxk5!ijW-\"a:/gXC?q`*JfxT x9s'0"yGɅ;|Fʖ?Z#7G-,S@̺TkPUe@ HO3}OmEX~!}_0L2*5GetUWo[< 3&t䨛pb;e= f$SƲr~1 q v.em^)ħeLJ$Hyƀ;r7TrDI\l˟ЊBx$Rb "$a99lԗ %O۬/2;r.ewN,f4^3̊_E]u] 1z" `ذK2V7Ė'uY:򹹳!uqs()dS$al 9GϋF~Y] 0BjIwñOΕ.TXS~gI ɃGJ_y`FQ$`Dy7=V/c\]џexof0bxGN~H=LAşK6ޅ *Ƥs)Q˅i;_?+;M <'hLbG2 e(I86Z]6S>aĝ+~+O?Yrv{&AV ߍi}p]9gi:Vw`A(o ^KIm 05@=ɕ NS|JRpÍR!sT{ŝ\JP0 !ߘ-JųbU_:ˡ-ծIEzBg} 8Q2)aM+%-Y_n6hi0γ<}H1 Qpjn̉$MSi4Tfcrωnin9_`{)X-͟ %1s"s\B?ئS4{x\ gg*  ,EI4B^X9esRiV&)R1w,Ϩp /Ñ:݊SXFu < /Jg04\C n#{~WcxW|UKPd@N !-E -:{m7 P>h QL$`l.ƇLe1M k7O#aˁd-(ƁQ Sywa X>βuKn;ë.K ;̋VdW1I*]+/bLFt3*4*(lHSŒtP$/ijsJQqQf/V!=pѕEg s`CEv8]jb-7+!w R=I\tUk S$v;ވ6gS<|W[|q>߹!5cO8؃FE6+ȸx{2~#M[4+S!@e9f.<=*f1 r`ipxQsVgruKr #+ft9K̶OO %7Wi&*h@n &y}+h5Ҷ#Zkhi&E`;eY$)DC쀭DhgNHC4dùIz._V_Ul}pq 6Ł6/<qOuE#~+Ej"Nq GqɣCOh˕.NZoC4Cp,tSPJ2A^BQA=Y>{Ni/2 N5smι9]^n=жui[I^0- Pӡ9ƚ MS Oj'jנV,1LJ(!ei0llSS]9'~/5}%8^Ŀ[ :pӞ$jI+xr!O)즛~RŤq[6 L *kg^6b 2H0jfM$ M_t;\'f-F0xQOt'#o]ݦQV\ICΕ__ p\'4Lp+@?=C:w&np:T{Qx4uҐVf,@N{]+}R_JOȝ:| SoɎ/@)|K!3-ۨ2F f2ڟ0M}^y}TM9`9y- SRpTA{*@޶%A#c)h0 d zr x Rq#rW~ )oWAC9i <<`3o -&'v/w;.^X=mDxiU6ÒT+rQQ:Fv[1?q/Ajێ)h6F8.?|3k-l1[ZH[kC&TWTPr/҃ޚf7o_c$*w1kv@ĒR ?8@tQ&ަE| {yvOA: FHd2 1!YwVԴZ 8~֊&*Hb<"ՉvΉ囄>98c<:t**k9P530Zn(D! @o~˩\6e´#3V!ty?z߬6o{W'&)rb*0=QD[v2=ͻͣC>QaS v4^ ^>qd{.G'97Z5.}N@AeuE;B5f#8,S`Vzcfzg^is9V2ح,{P$4Bq$Tڿ'2MKG0<+$`+x`ĪEL1a5& fϠ=Qe#Ai i["]4ˊ&_XM qHsN'hqO6q<- vzdܥ4cvnqWR <<K"&G;} ?Tٍx^vtǡnW 7:2& K('8-OlwIE⦶l)͈YH-뻲]5r.C.>?X$MaI ,Ukhf4+ɕ"}`i<&L^M:L?@x\QD?I|gw'Uu񖇰)ّz\|1arXBcB0H|u HP~=L0]eþt{$T|ßj~(iہCF \m:},Q')L`XH?f%gPb`L$ӹaЦK/%D"*} jPZD =EOLH?{{]x6RN-!Vюoz<ԘCQ P)1,k(o78R~C'=KWQad6 p†{WpI{U<ҕFwwVs٩pj"PDUV|y !IJS߇CN0 w2ԋѾGl?Q_CfHy|õe_iHOj~B\[[:UC/RDrxSh=~bpA@#w!oFLsr՘t&nb֔qPdDՍIR84nրf|e8{{k6Xę=Q*6{m#_W YaHdjH-0 rP@p.^fc7Ndi0nTYА2_:%4Kf%"R%=Z0SDxsw`J+꼮a(+IMѡy?vl-uj*SL`PFdU*l]nrANX:*IQRd&1xV68&X9b2v:l.lOE0*cޠ2 COpl>qu]UAMXWRqH*Yv;Rt^T p͵h̯ugvYM#@q`CaϞ+Ŀ-4a_`O{Qw\mVێ:J'i÷EU/PB,+tuk|~W1 `M_E6Z??i@V=qXO>;p{Yٶ߾ _ˮJ kWUY!@ Ecy~T@΋˼Trxꠀkôs~͏c8@ӹ޻}Ys~&mp'6MU9:FТ m_3Y`7A*ĞCN5} nֲl69F Cۢ]DQ6v.5 6`Z]UObNE#ד 5#/ SuLWn/ȷwӇRU?oic@j>@g|ؐC ÷ABKj″EF&Q*Gh^OҜ<#*vSKPZSlZD&cf6Q  Ĥ"Q͝H1O*^=k^lY 8lS~;K>ZU>W`r^`βbso:SD*f"yp3kἦrFK)0J)Ae %ߣڀ$yV\i{)rn嵹>e=C*3ClhہGFcX׉Ψ<42*IyhZ ?/ Bd]`^d)wO#l2JumF|S ~OQ 9&nb=:  7YEjp,y{g iha(l"|` L3S$Ui!wzjDiSa$Mvᬸuuf8Zq&wm_8SQnH[A&\ *HPՎsxiMoY5XM"@[̮UmIgy_ę4] X1m8Nnk"/J&K\53?xUyY2DPڄD$.E{i~mQ9O{|X./lY-V&a<1˺J1UYD|ؙjsζ% =(t<)b]_UUp!PSsE)_w> (@FLA>UnC^P,&f1DUAR"ol\1Mz PnGz5)maA~:\2;?3˗P-YpgHG=͹$N0'򅚍V`v-{P P.~~ɇqNVc|# astqO7vJtʉ@14׆v#M9"(&hWM5:]b*QcM)CGx=tY{S`5! ` ceX{ S"WS̽TCU]X [+u 35{ 轰dnD{Ο,v,ڭ*$yȡqN9B]:JG) bぉD2'Ǵ{D$&$a6r2Xg]7YoV X?@.zxʸ+9}-Dge,'R!x+gNtOUF3'j NnVA 47(q&DE0%&Yxożbg!^:M$"PpM}*e+@͔!˜t 5X"c˞_-`(ҍZ KqB *. DIkٯlhMy6sI!%g$F v=XGsy$>>k1N {)u+iᕺ5$ICx_«aO44d7Z޿HK(T){Z2 H3YX AfwE * _v;&g]dּOK>gO~ Vd1T.6x˄T*ߣ@n܎<$: (L]-v3gެڥ,tpǛÏ^-S&i?k^s#ᄃ]Iy':hV({e |S!;(ؤƩь ٢UD;3.5A =R`W~ޓp)o)h4{܃p8?M6 3Dri'qhmPv$g),V<<PRwv܏ ">:"yF 9䁋hB?{KdN'AWMx1|L1IXv5ǥS OR R)z$3΂h3ZF˺rMf勉=Xx7> [!N$t 3 :F^/3TJO^s|ĤkXw?_Ek\xߧߺ0󒾶x J'1ckW@"ivci0_g韢rX|1LLp=5-ŔFjJIZ*SLrtOK54Vi ݹ0Ɩ|T Cʜ@KC13,p p}rm7%;d;: jJoM׭nZPmfs2&+OI?)l]jʞ8˺92/VKX;d`A+ʢ߸go2_1ΑGiі>NOR{[Ehӂ3 F,gbSRiSu^ϵ&WKr07Ha\jx?<}{!1B ]z_HjT!R` hsWq 7CE1(ⓞwoLydXIPz[p}b9 :[&/RbHL(Z9x): nXznMkNd88}TPMuQ ppqnIef [1jXֵ۔`VV~''OCyKxW%\vs kDFGSJ<12;2KH޻F 2!vcW$g~V. 6%lLMIv|딙uR 4'˝|O:i֫QPCV:5![* j~a 8^9ťG,H +ɉ9Gudr8eb&܈/ϴkH4ŞSvŋ{hwUf*(O:v"YjbG>TDoG{vxlV+R` $1K 1^$5ۑ _sTɅSK[@moCL4[pHG5ް7LFa56%猪c/2;9~D.qTr|TeZ:|+yk?&?I0b qW/o `ҠvHjrGW7_7颏F&棈Z]K`:]NTr_˰gD=Y{ K_C&7]Xp-D$M* /XNS/B4̴.OoѮMp,N֋ۂzQ`=i&Y_AwE8CfU6ub!{g%>ٳ0lr Ow-jfU3pz%f҈9 n)X _̬V IeuNX-`4Zgج|VY`bo s"8gjLljY%@V2,R9Zx<-D9a-OA$m <34fm޿|Du u"Rb)mnv 2Y&[u=\G?`\& +@SP$fS̥|X\z>:7W`{.A,}{9j.$^DC c* 06NN$`%Ql\|J56WS]d֖B,ɕJ u!)Vwp^4BMBxEئ\WH>؉g *Y]_JiCj A吟.AOSTo܌n,Ϸ64FW:o#I&([ Ypb$G1ĩc"Ž$icŔ?_v\5oP~,nM·D f< 6CIH@t+P@Ґ4v<^};RFo /oNǼM$Uu;;MuV_ 1(""1'g1xLgP2Wduv(a+bPkGR z(ޫ0h4=bw7G i"d63]7q,'޲'3.\ z){pAud 45N/0^Z>ಇ>'y`B6IɣZʬ25NJ8P2 +E.*I|ʧ<f뤄x8)C8I_0?Q h^m~i?j"h IO]dxNcH{x"0" j@vmTdѮ !|U]oOf aUuEVIċ )q$TO'W։EodRj^j {$YӦFVw0 ¿ ikwI8/xvp_1J$K^h*/5k03L|MaQ[2iW+U}wNώLt*V$`u o_͔IPKX##[KBtr W3_s wŵg8=oel$#+C(~f''=LT VtL|7ݖ.'6{1N=hAbaYN7QtW |Á&>>()+צ?7r&bP\6_Rk&{f"ݵ8lpPn̆µU#q4U2)$ ܱ6J%i jN@=JXgÍs0 Y\|{K=%ٶ"DžTjF,HoUz>q^Ӝkƶ+hC5ʶF:NF@ӱ61%riQlO[냫޾'Q]WJI$[a_mX)[[_E hmzuAXbSjTh{f5_Wu9>eov~^~Nb!p }߲3ٻRxiܗѪ-_0+nԉA8;cSFB&`лdK烐S:|;y%Z4xNkℵ0tF@6jˀHPuQ] hT"v[ q8B'w/ﴣhA&=wR\5]w#;qڿb!n?W|.{[bIzHHf{[Ò 'нPJ$S@Ti-mt/Yt0+X$݀ؓ.r 4 ئL/)(2L be }'/Kl<#ӡ[w|;{Ρj~.cbg(dŽRL)nR^  Wll -&"-5wiZ@I XY`?blYOK[Gr [/~VzSaD15~l=Ab.nO2և8$ÜG!~o/< .9p{Eg)PN4UVsu5.(9kn q]'|R-"8U Ze?`-&w [["VhG;@@Qf NeS% 6ø.8TWيODP>Qe`X2`C0Q_-8ɧY]Sn  8R ڣTEIί'z_s$BCRѾCVV5%RlYHBIwxv}eB\}SM=jnT` olsdŧa>:W+-cZ?b鿭A[nLY+P5&X)^B@hC"Ǔ*l,YW8~#>Lޚ*9㎴o+, V(%a ͖&"D օl0&{O'O",(aqGgGqrXjgHZHz ^'# m[R!RAtx`;OcMRšmZh⴬OAx! [{mg1xnfM].cn+*֠4 Ǩ:gҰ6qa5|m;9Po'~t@BpYdҘ!b(Rtr @Qp}qRU*OV{EvY #U5( Af~j|Tjpt{MsmrX8Wst'Z=vcCDC̝ݎJaI!8^'\GK&E(fg &R%O(O%=5N^xJ7p1~ZrG#k/3#6AJı/\*Wv'V}}4mp^`&Q*+Nfg*ąaN-Ώ]DYя'V)1瘄"Ӻm,W(n_$ \y'.Jr=smtk_+࿽0mH*Ȉ§HK1j3 M W6KRpG-tr}[$lPwa֦%Cs[q4#QBI YKbXx^" ~;0bQL F8A, ӱAajK j[\N3WieT~6Jh~T@|%=ճc(R2v3)fl#;f=O>*AhL>0^Zݖ~Ca{L ac/MS 2(uW4cl wVYc/aJ riX ѩM\96-f!{uH<5z1!{“fQFY@tk-i19 Y$)XbޜuʙҐÛFk^FUQ1 b #J/ը7{Ʉ{~q=g\rk5pQ5_UOp~.*n%wD&e)up\עuLI%ٷ`k)]ot"p֢mPO1+|/l&憒$k_Ϥ soN9qˆ!_>_lv>kG.4N8EUSGj4PPGt&ʝN܇BÃ_KNLHy[?Z'dމplɆyEA} 'w1SQgrkFVǓu\%%JCz? TaKOB*kt1zt ◢ <8'WzX^P4gKxraaqWq~9% OZFoNnڤHKG}ͯ[fh2+sT34y!Y\[PJIK*Re IlWm\[޼< l1h@ ⌎'0B6~:vuڗ?h@0#L_t{"D' @Kl- ftX$kb:kU1:^SiE5'#O&D3ۊ$g2Lȱ3+]*eTS҅e񷺽YO{z=a9ds:jj߱Uk4[_F` &pI@rbvN]揭{/lx䯮fR7 ddQ>M`9ΑShݙW sQQ:VrgjĽ%0Xˮ Hb FhS-2Jo"0Pm| Zb֍Jcb5<D`AzEfі#e⧒ȸ6Ȕ1xd9\ks.6.`dbGa[[PҐv fPsيm[}xyhn5o@$?%iӌB^?0˭Ǔa^w=cUH6a‡S⵾*g (vp$ QKmo1n*?Ru,%-6>8Sp>Z혺4gkl)+_2E_BTd2 |P?5h ] ,f{ H rX 7ƕk:}E,4j ˯W﬇0ܤ5`O2Cjn}F*o\ O mqׂ9B$< c;-jb$i^]z&x!K߽<|;(..+%uJJ"sIS0`n`IՐDɼeAs/UVgX $ rg衩Uő?ު9-I ~x\xϳHّf&Neg¹+yBzc%s`[cgK3M X X6z ^D rr6Gb^61Ih1bǤ%/t>;Od"f U#eѬ?O#TX\ua3N1O2i7[QmbZD)`zRle8_~04lG@2q-ݾOq/s9KIiBg-(W-F/Oի˙6_s Zm߃#X3gn&nO`_i󜄌p9e/wf8wy_B*v(+\CjiK:kglAdt<93n,usd~5wfuȃOދ1}+UࠂkwaK85IU7$[pC{@ G7<oazc32K[uqApJ*$c[eƾ.u/58zSHڙѱFTŏy/qF\Bt$xFElou<{JځՌ1܈`Cζ,II7MJGqJ @rHRנ-xA]Olg )2T|0=Hk>8“5+>/`L3>/ms՛?7teUՖ;W+3L7sBo!s2Z0սk͌<)REؘ8=q䌀b@ 5Jm4\Imف*_`:jһ(%UK a.#rHpל"j5qq j 7h(vp܃dw^YYSt_ҥYo?NCɈ<*F *b]0TDPcpX%jCh^nۅ) ]q&qXk N":c|\SPn*_ZT/Ƈ( ys&.qj$Y&@cW?.rmDdv".a mFi2,lrW%"xIOr7*tç'a.+4U.$bJ PYP`3¡:Äi '> x rpgfnal/d ,a.P\ 'xӕDzsŌ !A=3PxLHBn)ue3qRtN0VVF?mL\d UJK"ۮ0ΑOBtz<8oV)NJ;ڐ/4Id!"_"8tR%X]RZחR_[[F)uKHO ~"~ ʟEwM )6P{= Mk:1ZO5^X`~*AU5[hT3an9?# ׶n(_@q!L:tC'W ɼЋ YZ