首頁 > 軟體

PostgreSQL索引失效會發生什麼

2022-09-20 22:03:57

前段時間碰到個奇怪的索引失效的問題,實際情況類似下面這樣:

bill=# begin;
BEGIN
bill=*# create index idx_t1 on t1(id);
CREATE INDEX
bill=*# explain select * from t1 where id = 1;
                     QUERY PLAN
----------------------------------------------------
 Seq Scan on t1  (cost=0.00..25.88 rows=6 width=36)
   Filter: (id = 1)
(2 rows)

bill=*# end;
COMMIT
bill=# explain select * from t1 where id = 1;
                             QUERY PLAN
---------------------------------------------------------------------
 Bitmap Heap Scan on t1  (cost=1.50..7.01 rows=6 width=36)
   Recheck Cond: (id = 1)
   ->  Bitmap Index Scan on idx_t1  (cost=0.00..1.50 rows=6 width=0)
         Index Cond: (id = 1)
(4 rows)

很顯然的問題就是,我在事務中建立了索引,卻沒辦法使用。但是當事務提交了後便可以正常使用了,這是什麼情況呢?

這個其實和pg_index中indcheckxmin屬性有關,關於這個欄位的解釋如下:

If true, queries must not use the index until the xmin of this pg_index row is below their TransactionXmin event horizon, because the table may contain broken HOT chains with incompatible rows that they can see

經檢查也確實如此:

bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass;
 indcheckxmin
--------------
 t
(1 row)

那麼問題來了,什麼情況下建立索引時會將索引的該屬性設定為true呢?

從前面官方檔案對於該欄位的解釋,如果表中包含broken HOT chains 則會為true,那什麼是broken HOT chains ?似乎和HOT機制有關。那是不是隻有存在broken HOT chains 才會設定為true呢?

這裡就不賣關子了,直接給出結論,然後我們再去一一驗證。

經測試發現,以下兩種情況會導致索引的indcheckxmin設定為true:

  • 當前事務中表上存在broken HOT chains,即官方檔案中所說;
  • 當old_snapshot_threshold被設定時。

場景一:broken HOT chains

這種情況,只要在當前事務中表中存在HOT更新的行時就會存在。那麼什麼時候會進行HOT更新呢?兩個前提:

  • 新的元組和舊元組必須在同一個page中;
  • 索引欄位不能進行更新。

既然如此,實際中常見的兩種情況就是:

  • 對錶上最後一個page進行更新;
  • 表設定了fillfactor,即每個page上有預留的空閒空間。

例子:

表中插入10條資料,自然只有1個page:

bill=# insert into t1 select generate_series(1,10),md5(random()::text);
INSERT 0 10

進行更新:

bill=# update t1 set info = 'bill' where id = 10;
UPDATE 1

檢視發現的確是HOT更新:

關於t_infomask2欄位的解釋這裡就不再贅述。

接下來我們建立索引:

可以發現indcheckxmin被設定為true,在當前事務中索引不可用。

經過驗證,在index_build階段,判斷到BrokenHotChain,便將indcheckxmin修改為true。

具體的修改程式碼如下:

/*此時indexInfo->ii_BrokenHotChain已被修改為true */	
if ((indexInfo->ii_BrokenHotChain || EarlyPruningEnabled(heapRelation)) &&
		!isreindex &&
		!indexInfo->ii_Concurrent)
	{
		Oid			indexId = RelationGetRelid(indexRelation);
		Relation	pg_index;
		HeapTuple	indexTuple;
		Form_pg_index indexForm;
		pg_index = table_open(IndexRelationId, RowExclusiveLock);
		indexTuple = SearchSysCacheCopy1(INDEXRELID,
										 ObjectIdGetDatum(indexId));
		if (!HeapTupleIsValid(indexTuple))
			elog(ERROR, "cache lookup failed for index %u", indexId);
		indexForm = (Form_pg_index) GETSTRUCT(indexTuple);
		/* If it's a new index, indcheckxmin shouldn't be set ... */
		Assert(!indexForm->indcheckxmin);
/*將indcheckxmin修改為true */
		indexForm->indcheckxmin = true;
		CatalogTupleUpdate(pg_index, &indexTuple->t_self, indexTuple);
		heap_freetuple(indexTuple);
		table_close(pg_index, RowExclusiveLock);
	}

同樣我們也可以驗證得知,的確是因為brokenhotchains導致的indcheckxmin被設定為true。

場景二:old_snapshot_threshold

先來看例子:

最簡單的場景,完全的一張空表,在事務中建立索引indcheckxmin就會被設定為true,果然索引也是不可用。

bill=# drop table t1;
DROP TABLE
bill=# create table t1(id int,info text);
CREATE TABLE
bill=# begin;
BEGIN
bill=*# create index idx_t1 on t1(id);
CREATE INDEX
bill=*# select indcheckxmin from pg_index where indexrelid = 'idx_t1'::regclass;
 indcheckxmin
--------------
 t
(1 row)

bill=*# explain select * from t1 where id = 1;
                     QUERY PLAN
----------------------------------------------------
 Seq Scan on t1  (cost=0.00..25.88 rows=6 width=36)
   Filter: (id = 1)
(2 rows)

那麼為什麼old_snapshot_threshold會產生這樣的影響呢?

經過跟蹤發現,當開啟該引數時,在事務中建立索引的snapshotdata結構如下:

(SnapshotData) $6 = {
  snapshot_type = SNAPSHOT_MVCC
  xmin = 856
  xmax = 856
  xip = 0x00007fd55c804fc0
  xcnt = 0
  subxip = 0x00007fd55ad5d000
  subxcnt = 0
  suboverflowed = false
  takenDuringRecovery = false
  copied = false
  curcid = 1
  speculativeToken = 0
  vistest = NULL
  active_count = 0
  regd_count = 0
  ph_node = {
    first_child = NULL
    next_sibling = NULL
    prev_or_parent = NULL
  }
  whenTaken = 691752041261069
  lsn = 208079736
}

而禁用該引數呢?

(SnapshotData) $7 = {
  snapshot_type = SNAPSHOT_MVCC
  xmin = 828
  xmax = 828
  xip = 0x00007fad31704780
  xcnt = 0
  subxip = 0x00007fad3155d000
  subxcnt = 0
  suboverflowed = false
  takenDuringRecovery = false
  copied = false
  curcid = 1
  speculativeToken = 0
  active_count = 0
  regd_count = 0
  ph_node = {
    first_child = NULL
    next_sibling = NULL
    prev_or_parent = NULL
  }
  whenTaken = 0
  lsn = 0
}

可以看到,區別在於不使用該引數時,建立snapshotdata不會設定whenTaken和lsn,那麼這兩個引數是幹嘛的呢?

先來看看snapshotdata的結構:

typedef struct SnapshotData
{
    SnapshotType snapshot_type; /* type of snapshot */
    /*
     * The remaining fields are used only for MVCC snapshots, and are normally
     * just zeroes in special snapshots.  (But xmin and xmax are used
     * specially by HeapTupleSatisfiesDirty, and xmin is used specially by
     * HeapTupleSatisfiesNonVacuumable.)
     *
     * An MVCC snapshot can never see the effects of XIDs >= xmax. It can see
     * the effects of all older XIDs except those listed in the snapshot. xmin
     * is stored as an optimization to avoid needing to search the XID arrays
     * for most tuples.
     */
    TransactionId xmin;         /* all XID < xmin are visible to me */
    TransactionId xmax;         /* all XID >= xmax are invisible to me */
    /*
     * For normal MVCC snapshot this contains the all xact IDs that are in
     * progress, unless the snapshot was taken during recovery in which case
     * it's empty. For historic MVCC snapshots, the meaning is inverted, i.e.
     * it contains *committed* transactions between xmin and xmax.
     *
     * note: all ids in xip[] satisfy xmin <= xip[i] < xmax
     */
    TransactionId *xip;
    uint32      xcnt;           /* # of xact ids in xip[] */
    /*
     * For non-historic MVCC snapshots, this contains subxact IDs that are in
     * progress (and other transactions that are in progress if taken during
     * recovery). For historic snapshot it contains *all* xids assigned to the
     * replayed transaction, including the toplevel xid.
     *
     * note: all ids in subxip[] are >= xmin, but we don't bother filtering
     * out any that are >= xmax
     */
    TransactionId *subxip;
    int32       subxcnt;        /* # of xact ids in subxip[] */
    bool        suboverflowed;  /* has the subxip array overflowed? */
    bool        takenDuringRecovery;    /* recovery-shaped snapshot? */
    bool        copied;         /* false if it's a static snapshot */
    CommandId   curcid;         /* in my xact, CID < curcid are visible */
    /*
     * An extra return value for HeapTupleSatisfiesDirty, not used in MVCC
     * snapshots.
     */
    uint32      speculativeToken;
    /*
     * For SNAPSHOT_NON_VACUUMABLE (and hopefully more in the future) this is
     * used to determine whether row could be vacuumed.
     */
    struct GlobalVisState *vistest;
    /*
     * Book-keeping information, used by the snapshot manager
     */
    uint32      active_count;   /* refcount on ActiveSnapshot stack */
    uint32      regd_count;     /* refcount on RegisteredSnapshots */
    pairingheap_node ph_node;   /* link in the RegisteredSnapshots heap */
    TimestampTz whenTaken;      /* timestamp when snapshot was taken */
    XLogRecPtr  lsn;            /* position in the WAL stream when taken */
    /*
     * The transaction completion count at the time GetSnapshotData() built
     * this snapshot. Allows to avoid re-computing static snapshots when no
     * transactions completed since the last GetSnapshotData().
     */
    uint64      snapXactCompletionCount;
} SnapshotData;

如上所示,TimestampTz表示snapshot何時產生的,為什麼啟用old_snapshot_threshold時會設定該值呢?

因為該值正是用來判斷快照是否過舊的:

/*
 * Implement slower/larger portions of TestForOldSnapshot
 *
 * Smaller/faster portions are put inline, but the entire set of logic is too
 * big for that.
 */
void
TestForOldSnapshot_impl(Snapshot snapshot, Relation relation)
{
        if (RelationAllowsEarlyPruning(relation)
                && (snapshot)->whenTaken < GetOldSnapshotThresholdTimestamp())
                ereport(ERROR,
                                (errcode(ERRCODE_SNAPSHOT_TOO_OLD),
                                 errmsg("snapshot too old")));
}

這樣我們也比較好理解為什麼設定了該引數時建立的索引在當前事務中不可用:

因為我們不設定該引數時,在事務中建立索引是可以保證MVCC的一致性,那麼索引便是安全可用的。

而使用引數時,由於TimestampTz被設定,資料庫會對其進行判斷該行資料是否已經過期,如果過期了那便會被清理掉,這樣對於索引來說便是不安全的,沒法保證資料的一致性,對於不是hot-safe的索引,自然要將其indcheckxmin設定為true,防止在事務中建立索引後資料實際已經過期被刪除的情況。

 /*
     * At this moment we are sure that there are no transactions with the
     * table open for write that don't have this new index in their list of
     * indexes.  We have waited out all the existing transactions and any new
     * transaction will have the new index in its list, but the index is still
     * marked as "not-ready-for-inserts".  The index is consulted while
     * deciding HOT-safety though.  This arrangement ensures that no new HOT
     * chains can be created where the new tuple and the old tuple in the
     * chain have different index keys.
     *
     * We now take a new snapshot, and build the index using all tuples that
     * are visible in this snapshot.  We can be sure that any HOT updates to
     * these tuples will be compatible with the index, since any updates made
     * by transactions that didn't know about the index are now committed or
     * rolled back.  Thus, each visible tuple is either the end of its
     * HOT-chain or the extension of the chain is HOT-safe for this index.
     */

總結

當pg_index的indcheckxmin欄位被設定為true時,直到此pg_index行的xmin低於查詢的TransactionXmin視界之前,查詢都不能使用此索引。

而產生這種現象主要有兩種情況:

1. 表上在當前事務中存在broken HOT chains;

2. old_snapshot_threshold被設定時。

到此這篇關於PostgreSQL索引失效會發生什麼的文章就介紹到這了,更多相關PostgreSQL索引失效內容請搜尋it145.com以前的文章或繼續瀏覽下面的相關文章希望大家以後多多支援it145.com!


IT145.com E-mail:sddin#qq.com