Date: 2023-08-17
Tag: 2.11.1

## Overview

2.11.1 is the 2nd [stable][release_policy] version of the 2.11 release
series. It introduces 12 improvements and resolves 73 bugs since
2.11.0.

The "stable" label means that we have all planned features implemented and we
see no high-impact issues. However, if you encounter an issue, feel free to
[report it][issues] on GitHub.

[release_policy]: https://www.tarantool.io/en/doc/latest/dev_guide/release_management/#release-policy
[issues]: https://github.com/tarantool/tarantool/issues

## Compatibility

Tarantool 2.x is backward compatible with Tarantool 1.10.x in the binary data
layout, client-server protocol, and replication protocol.

Please [upgrade][upgrade] using the `box.schema.upgrade()` procedure to unlock
all the new features of the 2.x series.

[upgrade]: https://www.tarantool.io/en/doc/latest/book/admin/upgrades/

## Functionality added or changed

### Core

* The `_schema.max_id` field in the `box.space._schema` function
  is now deprecated. Now the `_space:max()` function is used
  instead of `_schema.max_id` (gh-5997).
* Added the new function `box.malloc.info()` for reporting memory usage of
  Tarantool internal data structures allocated with `malloc()`. The function
  is available only on Linux (gh-7311).

### Replication

* Implemented correct recovery of mixed transactions. To do this, set
  `box.cfg.force_recovery` to `true`. If you need to revert to the old
  behavior, don't set the `force_recovery` option (gh-7932).

### Lua

* Embedded the `tarantool/metrics` module for metrics aggregation
  and export (gh-7725).
* Metrics can now be configured using the `box.cfg()` function (gh-7725).
* Added the `:is_new()` and `:is_old()` helpers to `tarantool.compat`
  options to simplify effective value checks (gh-8807).

### Box

* Allowed truncation of temporary and local spaces in the read-only mode
  (gh-5616).

#### Cfg

* Implemented a way to set a table as box.cfg{} options value via
  environment variables (gh-8051).

### Test

#### Fuzz

* LuaJIT now can be fuzzed using a grammar-based fuzzer (gh-4823).

### Build

*  The `zstd` version was updated to pre-1.5.5 (gh-8391).
* Added `tt` weak dependency. The `tt` command line utility combines
  `tarantoolctl` and `cartridge-cli` functionality into one tool.
  This change is a part of pushing `tt` to the public (tarantool/tt#286).
* Hardening against memory corruption attacks is now enabled by default on
  FreeBSD (gh-7536).
* Added the CMake option `FIBER_STACK_SIZE` to set the default fiber stack size.

## Bugs fixed

### Core

* Fixed various bugs related to unsafe (i.e., from an unknown source)
  decoding and validating of MsgPack extensions (ghs-73).
* Fixed a crash in the `net.box` client when a remote space had
  a field type that the client version did not support (gh-4632).
* Fixed a bug when a space creation failed with a duplication error.
  The issue occurred if the explicit and implicit space IDs were mixed.
  Now the actual maximal space `id` is used to generate a new one (gh-8036).
* Relaxed the tuple format requirements on tuples passed as the page starting
  position to `index:tuple_pos()` or to the `after` option of `index:select`.
  Now, Tarantool validates only the key parts of the index being used and all
  primary indexes (gh-8511).
* Fixed a crash that could happen when preparing a crash report on macOS
  (gh-8445).
* Fixed an integer overflow issue in `net.box` (ghs-121).
* An `IPROTO_EVENT` packet now has the same sync number as the last
  corresponding `IPROTO_WATCH` request (gh-8393).
* Fixed a bug because of which a dirty (not committed to WAL) DDL record could
  be written to a snapshot and cause a recovery failure (gh-8530).
* Fixed a bug when a tuple could be inserted even if it violates a `unique`
  constraint of a functional index that has a nullable part (gh-8587).
* Fixed a bug when Tarantool failed to decode a request containing an unknown
  IPROTO key. The bug resulted in broken connectivity between Tarantool 2.10
  and 2.11 (gh-8745).
* Fixed the node writing an empty `00000000000000000000.xlog` file regardless of
  the actual vclock when interrupted during the initial `box.cfg()` call
  (gh-8704).
* Fixed a bug causing the `ER_CURSOR_NO_TRANSACTION` failure for transactions
  on synchronous spaces when the `on_commit/on_rollback` triggers are set
  (gh-8505).
* Fixed a bug causing the effective session and user are not propagated to
  `box.on_commit` and `box.on_rollback` trigger callbacks when the transaction
  is synchronous (gh-8742).
* Fixed a bug when MVCC sometimes lost a gap record (gh-8326).
* Fixed a bug when MVCC rollback of a prepared statement could break internal
  invariants (gh-8648).
* Now MVCC engine automatically aborts a transaction if it reads changes
  of a prepared transaction and this transaction is aborted (gh-8654).
* Disabled the backtrace collection feature on the AArch64 Linux platform
  (gh-8572).
* Fixed a crash that could happen when Tarantool is started in the
  [background mode](https://www.tarantool.io/en/doc/latest/reference/configuration/#confval-background)
  (gh-6128).
* Fixed the collection of fiber backtraces on the M1/M2 macOS platform
  (gh-8074).
* Fixed a bug that caused writing incorrect values into the `stream_id` field
  of xlog headers (gh-8783).
* Fixed a bug when a space that is referenced by a foreign key could not
  be truncated even if the referring space was empty (gh-8946).
* Now foreign keys from non-temporary to temporary and from non-local to local
  spaces are prohibited since they can potentially break foreign key consistency
  (gh-8936).
* Fixed a crash when a collation used by a space was deleted (gh-4544).
* Fixed a crash that could happen when Tarantool is compiled by `clang`
  version 15 and above with enabled AddressSanitizer
  (tarantool/tarantool-qa#321).
* Fixed a bug because of which it was impossible to set the `hint` option
  to `true` for TREE indexes (gh-8937).

### Memtx

* Fixed the ability for the `memtx_memory` parameter to specify a memory size
  less than required for the stable operation of the engine (gh-7389).
* Fixed a heap-use-after-free bug in the transaction manager, which could occur
  when performing a DDL operation concurrently with a transaction on the same
  space (gh-8781).

### Vinyl

* Vinyl space and index directories are now created on demand.
  They are removed as soon as they become empty (gh-8441).
* Fixed a heap-use-after-free bug in the Vinyl read iterator caused by a race
  between a disk read and a memory dump task. The bug could lead to a crash or
  an invalid query result (gh-8852).

### Replication

* Fixed a bug related to `box.info.replication[...].upstream` being stuck in the "connecting"
  state for several minutes after a replica DNS record change (gh-7294).
* Fixed an issue when the nodes synchronizing with
  a hung leader reported the leader as alive.
  This behavior led to the delay of the new elections (gh-7515).
* Fixed a bug that occurred on applier failure: a node could start an election
  without having a quorum to do this (gh-8433).
* Fixed a possible failure to promote the desired node by `box.ctl.promote()` on
  a cluster with nodes configured with `election_mode = "candidate"` (gh-8497).
* Fixed nodes configured with `election_mode = 'candidate'` spuriously detecting
  a split-vote when another candidate should win with exactly a quorum of votes
  for it (gh-8698).
* Fixed a crash when using transactions with the linearizable isolation level during
  a replica reconnect (gh-7991).
* Fixed a possible crash on bootstrap with `box.cfg.bootstrap_strategy = 'auto'`
  when some of the bootstrapping nodes were stopped (gh-8757).

### LuaJIT

Backported patches from the vanilla LuaJIT trunk (gh-8069).
The following issues were fixed as part of this activity:

* Fixed successful `math.min/math.max` call with no args (gh-6163).
* Fixed inconsistencies in `math.min/math.max` calls with a NaN arg (gh-6163).
* Fixed `pcall()` call without arguments on arm64.
* Fixed assembling of ``IR_{AHUV}LOAD`` specialized to boolean for aarch64.
* Fixed constant rematerialization on arm64.
* Fixed `emit_rma()` for the x64/GC64 mode for non-`mov` instructions.
* Limited Lua C library path with the default `PATH_MAX` value of 4096 bytes.
* The JIT engine was disabled by default on macOS platforms to improve
  the user experience. If necessary, you can enable it with `jit.on` (gh-8252).
Backported patches from the vanilla LuaJIT trunk (gh-8516). The following issues
were fixed as part of this activity:

* Fixed `IR_LREF` assembling for the GC64 mode on x86_64.
* Fixed canonicalization of +-0.0 keys for `IR_NEWREF`.
* Fixed result truncation for `bit.rol` on x86_64 platforms.
* Fixed saved bytecode encapsulated in ELF objects.
* Fixed `lua_yield()` invocation inside C hooks.
* Fixed memory chunk allocation beyond the memory limit.
* Fixed TNEW load forwarding with instable types.
* Fixed use-def analysis for `BC_VARG`, `BC_FUNCV`.
Backported patches from the vanilla LuaJIT trunk (gh-8825). The following issues
were fixed as part of this activity:

* Fixed `BC_UCLO` insertion for returns.
* Fixed recording of `BC_VARG` with unused vararg values.
* Initialization instructions on trace are now emitted only for the first
  member of a union.

### Lua

* The `box.stat.memtx` function is now callable, it returns
  all memtx statistics. The `box.stat.memtx.tx()` function
  is now equivalent to the `box.stat.memtx().tx` function (gh-8448).
* Exported the missing `AUTH_TYPE` key to `box.iproto.key` (gh-8574).
* Stripped the unwanted `IPROTO_` prefix from the `POSITION`, `AFTER_POSITION`,
  and `AFTER_TUPLE` keys exported to `box.iproto.key` (gh-8577).
* Fixed the xlog reader Lua module to show unknown row header fields. Before
  this change the xlog reader silently skipped them.

#### Http client

* Fixed a bug where a response body cannot be decoded (gh-8363).
* Fixed the `Transfer-Encoding: chunked` setting being enabled even if
  the `Content-Length` header exists for stream requests (gh-8744).

#### Netbox

* Fixed a heap-use-after-free bug in the function creating a tuple format Lua
  object for `net.box` (gh-8889).

### SQL

* The `ANY` field type now supports collation in SQL (gh-8070).
* Now the `ARRAY`, `MAP`, and `INTERVAL` values cannot be used as
  arguments in the `ORDER BY` clause (gh-6668).
* Fixed incorrect conversion of an integer greater than `INT64_MAX` or
  less than `0` to a decimal number during SQL arithmetic operations (gh-8460).
* Fixed an assertion when selecting tuples with incomplete internal
  format (gh-8418).
* Fixed integer overflow issues in built-in functions (ghs-119).
* Fixed a possible assertion or segmentation fault when optimizing
  `INSERT INTO ... SELECT FROM` (gh-8661).
* Fixed an integer overflow issue and added a check for the `printf()` failure due
  to too large size (ghs-122).

### Box

* Fixed a bug where `box.cfg.force_recovery` doesn't work when there are
  no user spaces in a snapshot (gh-7974).
* Fixed a bug when large numbers were encoded incorrectly by `msgpackffi`.
  It could lead to wrong select results with large number keys (gh-6119).
* Fixed a crash on an unknown option of the function (gh-8463).
* The `takes_raw_args` option is now handled correctly
  during downgrading (gh-8457).
* Fixed several memory leaks on tuple encoding failures (gh-7939).
* Fixed a bug (bad error message) in pagination related to the validation of the
  `after` position option of the `:select` and `:pairs` methods of space and
  index objects (gh-8716).
* Fixed the memory leaks caused by the multi-statement transaction errors in the
  space index building and the space format checking operations (gh-8773).
* Fixed the privilege check when using spaces with functional indexes and
  constraints (gh-7873).
* Fixed a bug in the box console implementation because of
  which the `language` parameter was shared between connected
  clients (gh-8817).
* Fixed the inability to insert an integral number into a double-formatted
  field (gh-7483).
* Fixed the invalid memory access in a corner case of a specialized comparison
  function (gh-8899).
* Fixed a crash when `box.iproto.override` was called with unconfigured box.
  Now, an error is raised instead (gh-8975).

### Console

* Fixed `console.local_print()` failing on non-string arguments, which led to
  some rare errors. For example, when connecting via tarantoolctl to cartridged
  tarantool with incorrect credentials, a cdata error was passed through the
  `local_print()`, which failed to interpret it (gh-8374).

### Datetime

* Fixed an error in `datetime.set` when `timestamp` is passed along with `nsec`,
`usec`, or `msec` (gh-8583).
* Fixed errors when the string representation of a datetime object had
a negative nanosecond part (gh-8570).
* Fixed a bug with buffer overflow in tnt_strptime (gh-8502).
* Fixed a bug raising a false positive error when creating new intervals with
  range boundary values (gh-8878).

### Msgpack

* Fixed decoding datetime intervals with fields larger than possible int32
  values (gh-8887).

### Build

* Enabled compiler optimizations for static build dependencies, which were
  erroneously disabled in version 2.10.3 (gh-8606).
