-
Notifications
You must be signed in to change notification settings - Fork 1.3k
FoundationDB Trace Metrics Definitions
Sam Gwydir edited this page Aug 27, 2020
·
4 revisions
FoundationDB outputs various messages to /var/log/foundationdb
. While some of these may be self-explanatory, others are hard to decipher. This page is a full listing of all metrics (as of v6.2.15 20566f2ff06a7e822b30e8cfd91090fbd863a393) and their definitions.
Initially these will be blank. If you come here to find a metric definition and do not find it -- please update the page when you do!
Metric Name | Description |
---|---|
Type | |
Time | |
Severity | |
Machine | |
LogGroup | |
ID | |
Roles |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588967689.669706 |
|
Type |
AsyncFileKAIOOpen |
|
ID |
0000000000000000 |
|
Filename |
/mnt/fdb/transaction/4519/logqueue-V_3_LS_2-ebfd05340047e0023ebfd5976921797d-0.fdq |
|
Flags |
983046 |
|
Mode |
1408 |
|
Fd |
76 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588947778.846806 |
|
Type |
BgDDMountainChopper |
|
ID |
47cedbda4d178447 |
|
SourceBytes |
4853627018 |
|
DestBytes |
3153236645 |
|
ShardBytes |
24260500 |
|
AverageShardBytes |
17832010 |
|
SourceTeam |
Size 3; 10.180.50.80:4505 266809676c603f74 10.180.50.50:4504 54a823d4ae7529d3, 10.180.50.179:4502 5ce25c2920b67979 |
|
DestTeam |
Size 3; 10.180.50.80:4500 1ddcb14a4183c131 10.180.50.179:4501 4c48cfbd569beedc, 10.180.50.59:4502 9cd1b1e104c9b375 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588947773.943168 |
|
Type |
BgDDValleyFiller |
|
ID |
47cedbda4d178447 |
|
SourceBytes |
3209342253 |
|
DestBytes |
2784814454 |
|
ShardBytes |
30515983 |
|
AverageShardBytes |
17832010 |
|
SourceTeam |
Size 3; 10.180.50.133:4501 0d0549ca42863d3d 10.180.50.77:4505 16c1de13480928f6, 10.180.50.59:4500 ba09eca3913bf7ea |
|
DestTeam |
Size 3; 10.180.50.179:4500 739f9922184049d5 10.180.50.59:4501 7b4181ea6025fff2, 10.180.50.77:4502 e8181de673760f68 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032655.510371 |
|
OriginalTime |
1589032652.301211 |
|
Type |
ClusterControllerMetrics |
|
ID |
71f062b5aa2e75b9 |
|
Elapsed |
5.00001 |
|
OpenDatabaseRequests |
1.2 5.11374 7659081 |
|
RegisterWorkerRequests |
0 0 1201 |
|
GetWorkersRequests |
0 0 0 |
|
GetClientWorkersRequests |
0 0 0 |
|
RegisterMasterRequests |
0 0 149 |
|
GetServerDBInfoRequests |
0.399999 1.52986 1845456 |
|
StatusRequests |
0 0 425054 |
|
FailureMonitoringRequests |
719.198 4.12408 5493283854 |
|
ServersFailed |
0 0 271 |
|
ServersUnfailed |
0 0 343 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589306127.517835 |
|
Type |
ClusterControllerWorkerAlreadyRegistered |
|
ID |
71f062b5aa2e75b9 |
|
SuppressedEventCount |
0 |
|
WorkerId |
6e9e2d06650b7805 |
|
ProcessId |
a907b89762a1511a65776582eb9c6add |
|
ZoneId |
d5e9f92c0f8335f87d059e9f83e731b3 |
|
DataHall |
3 |
|
PClass |
stateless |
|
Workers |
72 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032656.245691 |
|
Type |
ClusterGetStatus |
|
ID |
0000000000000000 |
|
Duration |
0.0272791 |
|
StatusSize |
85872 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589229311.314538 |
|
Type |
CodeCoverage |
|
ID |
0000000000000000 |
|
File |
fdbserver/storageserver.actor.cpp |
|
Line |
3432 |
|
Condition |
true |
|
Machine |
10.180.50.179:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.473401 |
|
Type |
CommitAttachID |
|
ID |
0345df4c2682866e |
|
To |
8af7049501fab007 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.404859 |
|
Type |
CommitBatchesMemoryLimit |
|
ID |
0000000000000000 |
|
BytesLimit |
429496729 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.472888 |
|
Type |
CommitDebug |
|
ID |
0345df4c2682866e |
|
Location |
MasterProxyServer.batcher |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589306127.517460 |
|
Type |
CommitDummyTransaction |
|
ID |
0000000000000000 |
|
Key |
\xff/SC/p\xcb\xd8\xb2t\\xb3;1v\xa4\xcf\xf9\x8c\xf7 |
|
Retries |
0 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624446.751595 |
|
Type |
CommitDummyTransactionError |
|
ID |
0000000000000000 |
|
Error |
operation_cancelled |
|
ErrorDescription |
Asynchronous operation cancelled |
|
ErrorCode |
1101 |
|
Key |
\xff/logs |
|
Retries |
0 |
|
Machine |
10.180.50.77:4518 |
|
LogGroup |
default |
|
Roles |
MS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589440609.340105 |
|
Type |
ConnectedOutgoing |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.50:4516 |
|
Machine |
10.180.50.179:4503 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589440609.338790 |
|
Type |
ConnectingTo |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.50:4516 |
|
Machine |
10.180.50.179:4503 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588950096.127790 |
|
Type |
ConnectionClosed |
|
ID |
2ca7bfdaeffe987a |
|
Error |
connection_failed |
|
ErrorDescription |
Network connection failed |
|
ErrorCode |
1026 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.77:4518 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588953503.058843 |
|
Type |
ConnectionEstablished |
|
ID |
3c385b767912e1d8 |
|
SuppressedEventCount |
0 |
|
Peer |
10.180.50.59:34208 |
|
ConnectionId |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589440609.339243 |
|
Type |
ConnectionExchangingConnectPacket |
|
ID |
cbe8783874023296 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.50:4516 |
|
Machine |
10.180.50.179:4503 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588953503.058843 |
|
Type |
ConnectionFrom |
|
ID |
3c385b767912e1d8 |
|
SuppressedEventCount |
0 |
|
FromAddress |
10.180.50.59:34208 |
|
ListenAddress |
10.180.50.179:4500 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620913.463151 |
|
Type |
ConnectionTimedOut |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.80:4516 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620801.277431 |
|
Type |
ConnectionTimeout |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
1 |
|
WithAddr |
10.180.50.80:4501 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.224096 |
|
Type |
CoordinatedStateSet |
|
ID |
0000000000000000 |
|
Gen |
72 |
|
Wgen |
72 |
|
Genu |
4132d40fe4b7295c |
|
Wgenu |
4132d40fe4b7295c |
|
Cgen |
72 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949856.367059 |
|
Type |
CoordinationPing |
|
ID |
99ae80c2014db2b1 |
|
CCID |
71f062b5aa2e75b9 |
|
TimeStep |
126270 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032656.518897 |
|
Type |
CoordinationPingSent |
|
ID |
71f062b5aa2e75b9 |
|
TimeStep |
127650 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1584124303.900535 |
|
Type |
DDInitGotInitialDD |
|
ID |
47cedbda4d178447 |
|
B |
\x15(\x02\x02\x05\x0d\x00\x00\x00\x00\x00\x07h\xd2 |
|
E |
\xff\xff |
|
Src |
014e19075c662a8f202b8a450ad604d42ad341d7b4caeb5e80b8975b6ef98d69,3821fc63e0d45b3238b4159ad69e2618 |
|
Dest |
[no items] |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588970360.114821 |
|
Type |
DDRecruiting |
|
ID |
0000000000000000 |
|
State |
Sending request to CC |
|
Exclusions |
36 |
|
Critical |
0 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1584124339.015333 |
|
Type |
DDTrackerStarting |
|
ID |
47cedbda4d178447 |
|
State |
Active |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1588935343.580889 |
|
Type |
DDTrackerStats |
|
ID |
47cedbda4d178447 |
|
Shards |
6787 |
|
TotalSizeBytes |
245391561136 |
|
SystemSizeBytes |
0 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949825.432928 |
|
Type |
DiskMetrics |
|
ID |
739f9922184049d5 |
|
ReadOps |
0 |
|
WriteOps |
15 |
|
ReadQueue |
0 |
|
WriteQueue |
0 |
|
GlobalSQLiteMemoryHighWater |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588967689.669706 |
|
Type |
DiskQueueReplaceFile |
|
ID |
ebfd05340047e002 |
|
Filename |
/mnt/fdb/transaction/4519/logqueue-V_3_LS_2-ebfd05340047e0023ebfd5976921797d-0.fdq |
|
OldFileSize |
27211550720 |
|
ElidedTruncateSize |
21211549696 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588967744.860605 |
|
Type |
DiskQueueReplaceTruncateEnded |
|
ID |
0000000000000000 |
|
Filename |
/mnt/fdb/transaction/4519/logqueue-V_3_LS_2-ebfd05340047e0023ebfd5976921797d-0.fdq |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.382418 |
|
Type |
DumpToken |
|
ID |
ef3357ccedce9d99 |
|
Name |
recruited.commit |
|
Token |
ef3357ccedce9d99 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620888.228939 |
|
Type |
EndpointNotFound |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
Address |
10.180.50.77:4516 |
|
Token |
0cc6c04fad8cdccf |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588951097.176744 |
|
Type |
FKBlockFail |
|
ID |
739f9922184049d5 |
|
Error |
operation_cancelled |
|
ErrorDescription |
Asynchronous operation cancelled |
|
ErrorCode |
1101 |
|
SuppressedEventCount |
0 |
|
FKID |
fa9327589855f12d |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588967689.672385 |
|
Type |
FSyncParentDir |
|
ID |
0000000000000000 |
|
Folder |
/mnt/fdb/transaction/4519/ |
|
File |
/mnt/fdb/transaction/4519/logqueue-V_3_LS_2-ebfd05340047e0023ebfd5976921797d-0.fdq |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589440609.332510 |
|
Type |
FailureMonitorClientSlow |
|
ID |
0000000000000000 |
|
Elapsed |
1.33938 |
|
Expected |
0.1 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588621263.584732 |
|
Type |
FailureMonitoringServerDown |
|
ID |
0000000000000000 |
|
OldServerID |
71f062b5aa2e75b9 |
|
Machine |
10.180.50.80:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588621263.886816 |
|
Type |
FailureMonitoringServerUp |
|
ID |
0000000000000000 |
|
OldServer |
71f062b5aa2e75b9 |
|
Machine |
10.180.50.80:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
30 |
|
Time |
1588970499.320355 |
|
Type |
FetchKeysTooLong |
|
ID |
0000000000000000 |
|
Duration |
600.003 |
|
Phase |
1 |
|
Begin |
mako065404742xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\x00 |
|
End |
mako0654306 |
|
Machine |
10.180.50.59:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589227564.994779 |
|
Type |
FetchPast |
|
ID |
4c48cfbd569beedc |
|
TotalAttempts |
1 |
|
FKID |
ede17487d7eb7ec2 |
|
V |
7856448270943 |
|
N |
7856448270943 |
|
E |
7856448270943 |
|
Machine |
10.180.50.179:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
30 |
|
Time |
1588951754.826600 |
|
Type |
FinishMoveKeysTooLong |
|
ID |
0000000000000000 |
|
Duration |
600 |
|
Servers |
673a4fea98c33d79bd4a92cfcf3ff0eca89027238077abaed423b976c88806d5,b0037425027a811f4ab8ac1b8e92510f |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.359413 |
|
Type |
FullSnapshotEnd |
|
ID |
d233af26b0ac14a9 |
|
PreviousSnapshotEndLoc |
0.595359 |
|
SnapshotSize |
1390 |
|
SnapshotElements |
1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.358701 |
|
Type |
GenerationRegReadReply |
|
ID |
0000000000000000 |
|
RVSize |
1351 |
|
VWG |
56 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.358701 |
|
Type |
GenerationRegReadRequest |
|
ID |
0000000000000000 |
|
From |
10.180.50.133:4518 |
|
K |
fdb_chaos_testing:8CYRO6j9 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.362043 |
|
Type |
GenerationRegWrote |
|
ID |
0000000000000000 |
|
From |
10.180.50.133:4518 |
|
Key |
fdb_chaos_testing:8CYRO6j9 |
|
ReqGen |
57 |
|
Returning |
57 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.223495 |
|
Type |
GetDurableResult |
|
ID |
610cf5205c59d595 |
|
Required |
2 |
|
Present |
1 |
|
ServerState |
fafff |
|
RecoveryVersion |
-1 |
|
EndVersion |
7252781666373 |
|
SafeBegin |
0 |
|
SafeEnd |
0 |
|
NewSafeBegin |
0 |
|
KnownCommittedVersion |
7252779663229 |
|
EpochEnd |
9223372036854775807 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.222850 |
|
Type |
GetDurableResultWaiting |
|
ID |
610cf5205c59d595 |
|
Required |
2 |
|
Present |
0 |
|
ServerState |
fffff |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588486349.726138 |
|
Type |
GetLeaderReply |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
Coordinator |
10.180.50.80:4516 |
|
Nominee |
622c813b77d23c79 |
|
ClusterKey |
fdb_chaos_testing:8CYRO6j9 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588952423.949376 |
|
Type |
GetMagazineSample |
|
ID |
0000000000000000 |
|
Size |
128 |
|
Backtrace |
addr2line -e fdbserver.debug -p -C -f -i 0x199945c 0x195f588 0x195f7d1 0x18bada4 0x18b2e32 0x18b5bb2 0x711564 0x746640 0x756378 0x7564e6 0x7565db 0x75668c 0xb5badf 0xb5d4fa 0xb5e10b 0xa38856 0xa353e9 0x189da86 0x7fbe95498140 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589306128.017844 |
|
Type |
GetTLogTeamDone |
|
ID |
0000000000000000 |
|
Completed |
1 |
|
Policy |
data_hall^2 x zoneid^2 x 1 |
|
Results |
6 |
|
Processes |
6 |
|
Workers |
72 |
|
Required |
4 |
|
Desired |
6 |
|
RatingTests |
200 |
|
PolicyGenerations |
100 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032664.326002 |
|
Type |
GetValueAttachID |
|
ID |
8cd1b5993774ae29 |
|
To |
c8343088d8219a7e |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588974947.417118 |
|
Type |
GetValueDebug |
|
ID |
42e97f74dbc76a76 |
|
Location |
storageServer.received |
|
Machine |
10.180.50.179:4503 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588950044.938079 |
|
Type |
GotServerDBInfoChange |
|
ID |
0000000000000000 |
|
ChangeID |
300f5203b4b3ba9f |
|
MasterID |
1bb55f8b45a73d07 |
|
RatekeeperID |
3ebe1a9f1f09a897 |
|
DataDistributorID |
47cedbda4d178447 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588951071.557801 |
|
Type |
HugeArenaSample |
|
ID |
0000000000000000 |
|
Count |
1 |
|
Size |
411480 |
|
Backtrace |
addr2line -e fdbserver.debug -p -C -f -i 0x1957f87 0x69c0ec 0x18deb12 0x19d8d88 0x663015 0x7fbe95472505 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588953503.058843 |
|
Type |
IncomingConnection |
|
ID |
3c385b767912e1d8 |
|
SuppressedEventCount |
0 |
|
FromAddr |
10.180.50.59:34208 |
|
CanonicalAddr |
10.180.50.59:34208 |
|
IsPublic |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620845.918082 |
|
Type |
IncomingConnectionError |
|
ID |
b13613d198545c6e |
|
Error |
connection_failed |
|
ErrorDescription |
Network connection failed |
|
ErrorCode |
1026 |
|
SuppressedEventCount |
0 |
|
FromAddress |
10.180.50.80:34600 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588951198.673726 |
|
Type |
KVCommit10sSample |
|
ID |
739f9922184049d5 |
|
Queued |
0.0461066 |
|
Commit |
0.0101311 |
|
Checkpoint |
0.168605 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.404859 |
|
Type |
KVSMemRecovered |
|
ID |
ef3357ccedce9d99 |
|
SnapshotItems |
0 |
|
SnapshotEnd |
0 |
|
Mutations |
0 |
|
Commits |
0 |
|
TimeTaken |
0 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.404859 |
|
Type |
KVSMemRecoveryComplete |
|
ID |
ef3357ccedce9d99 |
|
Reason |
Non-header sized data read |
|
DataSize |
0 |
|
ZeroFillSize |
0 |
|
SnapshotEndLocation |
0.1 |
|
NextReadLoc |
0.1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.404859 |
|
Type |
KVSMemRecoveryStarted |
|
ID |
ef3357ccedce9d99 |
|
SnapshotEndLocation |
0.1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.404859 |
|
Type |
KVSMemStartingSnapshot |
|
ID |
ef3357ccedce9d99 |
|
StartKey |
`` | |
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.359413 |
|
Type |
KVSMemSwitchingToLargeTransactionMode |
|
ID |
d233af26b0ac14a9 |
|
TransactionSize |
1417 |
|
DataSize |
2120 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CD |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589227424.258954 |
|
Type |
LayerStatusError |
|
ID |
0000000000000000 |
|
Error |
transaction_timed_out |
|
ErrorDescription |
Operation aborted because the transaction timed out |
|
ErrorCode |
1031 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032656.242950 |
|
Type |
LayerStatusFetcher |
|
ID |
0000000000000000 |
|
Duration |
0.00563216 |
|
StatusSize |
3022 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620801.624767 |
|
Type |
LeaderChanged |
|
ID |
62734ed33127b4f7 |
|
ToID |
002c813b77d23c79 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949829.583142 |
|
Type |
MachineLoadDetail |
|
ID |
0000000000000000 |
|
User |
57406479 |
|
Nice |
2418390 |
|
System |
39988960 |
|
Idle |
11992251032 |
|
IOWait |
2063309 |
|
IRQ |
0 |
|
SoftIRQ |
8985462 |
|
Steal |
37523 |
|
Guest |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949824.773677 |
|
OriginalTime |
1588949824.583129 |
|
Type |
MachineMetrics |
|
ID |
0000000000000000 |
|
Elapsed |
5.00001 |
|
MbpsSent |
2.72133 |
|
MbpsReceived |
2.14191 |
|
OutSegs |
14067 |
|
RetransSegs |
0 |
|
CPUSeconds |
0.0450846 |
|
TotalMemory |
133652574208 |
|
CommittedMemory |
52253396992 |
|
AvailableMemory |
81399177216 |
|
ZoneID |
60062d710b037cb0bf737e2d7278790d |
|
MachineID |
60062d710b037cb0bf737e2d7278790d |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588621409.821659 |
|
Type |
MasterCommittedTLogs |
|
ID |
880566f6d4ddee15 |
|
TLogs |
0: adcaec5a089596b096b5a18b206c9841 95d40059bdb8bfd0cba435415e532921, 70552e191c5685331d3016ddfff999e9, 4366e563e34a058e2b99e6246bf4e7a3, b3f6da478d8a9cf40e90bec614a71c68, c26b9bf32aac5c9ba76f3069a5b42b7f |
|
RecoveryCount |
45 |
|
RecoveryTransactionVersion |
7249738881113 |
|
Machine |
10.180.50.179:4518 |
|
LogGroup |
default |
|
Roles |
MS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.219712 |
|
Type |
MasterLifetime |
|
ID |
610cf5205c59d595 |
|
LifetimeToken |
71f062b5aa2e75b9#22 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620809.679012 |
|
Type |
MasterProxyTerminated |
|
ID |
ef3357ccedce9d99 |
|
Error |
tlog_stopped |
|
ErrorDescription |
TLog stopped |
|
ErrorCode |
1011 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588606950.607957 |
|
OriginalTime |
1586587214.484365 |
|
Type |
MasterRecoveredConfig |
|
ID |
c93a9d5aa846ac9b |
|
Conf |
`{\log_spill:2\logs:6, | |
edundancy_mode: hree_data_hall,\storage_engine:\ssd-2,\usable_regions:1}` | ||
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.235072 |
|
Type |
MasterRecovering |
|
ID |
610cf5205c59d595 |
|
LastEpochEnd |
7252781666373 |
|
RecoveryTransactionVersion |
7252881666373 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.219712 |
|
Type |
MasterRecovery |
|
ID |
610cf5205c59d595 |
|
BeginPair |
0c5e5d4863f3ba9a |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588621409.775470 |
|
Type |
MasterRecoveryCommit |
|
ID |
880566f6d4ddee15 |
|
Machine |
10.180.50.179:4518 |
|
LogGroup |
default |
|
Roles |
MS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588606950.607957 |
|
OriginalTime |
1586587214.555439 |
|
Type |
MasterRecoveryDuration |
|
ID |
c93a9d5aa846ac9b |
|
RecoveryDuration |
0.0903919 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588606950.607957 |
|
OriginalTime |
1586587225.075300 |
|
Type |
MasterRecoveryState |
|
ID |
c93a9d5aa846ac9b |
|
StatusCode |
14 |
|
Status |
fully_recovered |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.355769 |
|
Type |
MasterTerminated |
|
ID |
c93a9d5aa846ac9b |
|
Error |
master_tlog_failed |
|
ErrorDescription |
Master terminating because a TLog failed |
|
ErrorCode |
1205 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.226471 |
|
Type |
MasterUpdateRegistration |
|
ID |
610cf5205c59d595 |
|
RecoveryCount |
48 |
|
Logs |
anti: 0 replication: 4 local: 1 routers: 0 tLogs: 701adf5893b590716c046170438ae7704b24114eba5b6c6c161121f96c764bfa,5af8954c2e85d21d50cb9472a392c008,f77128b5c4256d72d9968c7d8d523c38,d4d6e33e212b440a214c51d4feda4319 locality: -1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949829.583142 |
|
Type |
MemoryMetrics |
|
ID |
0000000000000000 |
|
TotalMemory16 |
262144 |
|
ApproximateUnusedMemory16 |
0 |
|
ActiveThreads16 |
1 |
|
TotalMemory32 |
7208960 |
|
ApproximateUnusedMemory32 |
4325376 |
|
ActiveThreads32 |
1 |
|
TotalMemory64 |
216268800 |
|
ApproximateUnusedMemory64 |
166985728 |
|
ActiveThreads64 |
3 |
|
TotalMemory96 |
284880960 |
|
ApproximateUnusedMemory96 |
49533120 |
|
ActiveThreads96 |
1 |
|
TotalMemory128 |
219676672 |
|
ApproximateUnusedMemory128 |
114163712 |
|
ActiveThreads128 |
1 |
|
TotalMemory256 |
9961472 |
|
ApproximateUnusedMemory256 |
9306112 |
|
ActiveThreads256 |
1 |
|
TotalMemory512 |
7864320 |
|
ApproximateUnusedMemory512 |
7602176 |
|
ActiveThreads512 |
1 |
|
TotalMemory1024 |
7602176 |
|
ApproximateUnusedMemory1024 |
7340032 |
|
ActiveThreads1024 |
1 |
|
TotalMemory2048 |
9568256 |
|
ApproximateUnusedMemory2048 |
9306112 |
|
ActiveThreads2048 |
1 |
|
TotalMemory4096 |
5375131648 |
|
ApproximateUnusedMemory4096 |
49938432 |
|
ActiveThreads4096 |
1 |
|
TotalMemory8192 |
323354624 |
|
ApproximateUnusedMemory8192 |
322174976 |
|
ActiveThreads8192 |
1 |
|
HugeArenaMemory |
103468 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620801.624710 |
|
Type |
MonitorLeaderChange |
|
ID |
0000000000000000 |
|
NewLeader |
002c813b77d23c79 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620801.624767 |
|
Type |
MonitorLeaderForProxiesChange |
|
ID |
0000000000000000 |
|
NewLeader |
002c813b77d23c79 |
|
Key |
fdb_chaos_testing:8CYRO6j9 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588606950.634456 |
|
Type |
MonitorLeaderForProxiesGotClientInfo |
|
ID |
71f062b5aa2e75b9 |
|
Proxy0 |
85655d71658d880b |
|
ClientID |
a6b7ed2b5da6bee6 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1588935347.593266 |
|
Type |
MovingData |
|
ID |
47cedbda4d178447 |
|
InFlight |
0 |
|
InQueue |
0 |
|
AverageShardSize |
44982850 |
|
UnhealthyRelocations |
0 |
|
HighestPriority |
0 |
|
BytesWritten |
1854082563511 |
|
PriorityRecoverMove |
0 |
|
PriorityRebalanceUnderutilizedTeam |
0 |
|
PriorityRebalanceOverutilizedTeam |
0 |
|
PriorityTeamHealthy |
0 |
|
PriorityTeamContainsUndesiredServer |
0 |
|
PriorityTeamRedundant |
0 |
|
PriorityMergeShard |
0 |
|
PriorityTeamUnhealthy |
0 |
|
PriorityTeam2Left |
0 |
|
PriorityTeam1Left |
0 |
|
PriorityTeam0Left |
0 |
|
PrioritySplitShard |
0 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588620913.463151 |
|
Type |
N2_ConnectError |
|
ID |
50b3dd1421b256ae |
|
SuppressedEventCount |
0 |
|
Message |
125 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588950096.127790 |
|
Type |
N2_ReadError |
|
ID |
2ca7bfdaeffe987a |
|
SuppressedEventCount |
0 |
|
Message |
2 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588950095.007774 |
|
Type |
N2_ReadProbeError |
|
ID |
562454f4e1afb8be |
|
SuppressedEventCount |
0 |
|
Message |
125 |
|
Machine |
10.180.50.179:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589222264.244907 |
|
Type |
Net2SlowTaskTrace |
|
ID |
0000000000000000 |
|
TraceTime |
1589222264.383268 |
|
Trace |
addr2line -e fdbserver.debug -p -C -f -i 0x7fbe95d375f0 0x69c3d0 0xf648ca 0xfa933e 0xfaa7f7 0xf6b111 0xf69469 0xfad5ec 0xf69a19 0xf6a2ab 0xfada46 0xfb6cac 0xf8602e 0xf865eb 0xf871bb 0xf87697 0x69f938 0xf64ef3 0xf650f1 0xf72e8b 0xf72ffc 0x69f938 0xdd1a79 0xb1be48 0xf9fb54 0xb52a51 0x19d8d88 0x663015 0x7fbe95472505 0x68f2f5 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949824.773677 |
|
OriginalTime |
1588949824.583129 |
|
Type |
NetworkMetrics |
|
ID |
0000000000000000 |
|
Elapsed |
5.00001 |
|
CantSleep |
0 |
|
WontSleep |
0 |
|
Yields |
5 |
|
YieldCalls |
1434 |
|
YieldCallsTrue |
0 |
|
SlowTaskSignals |
0 |
|
YieldBigStack |
0 |
|
RunLoopIterations |
2590 |
|
TimersExecuted |
2920 |
|
TasksExecuted |
3479 |
|
ASIOEventsProcessed |
3888 |
|
ReadCalls |
1426 |
|
WriteCalls |
712 |
|
ReadProbes |
713 |
|
WriteProbes |
0 |
|
PacketsRead |
713 |
|
PacketsGenerated |
712 |
|
WouldBlock |
713 |
|
LaunchTime |
0.0011158 |
|
ReactTime |
0.00257254 |
|
SlowTask1M |
1 |
|
SlowTask2M |
1 |
|
PriorityBusy1 |
0.0246766 |
|
PriorityMaxBusy1 |
0.000935555 |
|
PriorityBusy2050 |
0.0246766 |
|
PriorityMaxBusy2050 |
0.000935555 |
|
PriorityBusy3050 |
0.0205729 |
|
PriorityMaxBusy3050 |
0.000935555 |
|
PriorityBusy4050 |
0.0205729 |
|
PriorityMaxBusy4050 |
0.000935555 |
|
PriorityBusy4950 |
0.0205729 |
|
PriorityMaxBusy4950 |
0.000935555 |
|
PriorityBusy5050 |
0.0189734 |
|
PriorityMaxBusy5050 |
0.000935555 |
|
PriorityBusy7050 |
0.014112 |
|
PriorityMaxBusy7050 |
0.000935555 |
|
PriorityBusy8050 |
0.014112 |
|
PriorityMaxBusy8050 |
0.000935555 |
|
PriorityBusy10050 |
0.00535178 |
|
PriorityMaxBusy10050 |
0.000935555 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.236639 |
|
Type |
NewEpochStartVersion |
|
ID |
610cf5205c59d595 |
|
StartVersion |
7252779663230 |
|
EpochEnd |
7252779663230 |
|
Locality |
-1 |
|
OldLogRouterTags |
0 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.335458 |
|
Type |
NotifyAddressFailed |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
Address |
10.180.50.80:4519 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620809.176447 |
|
Type |
NotifyAddressHealthy |
|
ID |
0000000000000000 |
|
SuppressedEventCount |
0 |
|
Address |
10.180.50.80:4519 |
|
Machine |
10.180.50.179:4514 |
|
LogGroup |
default |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.235072 |
|
Type |
PeekNextEnd |
|
ID |
0000000000000000 |
|
Queue |
0 |
|
Bytes |
12 |
|
Loc |
7252781666374 |
|
End |
7252781666374 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.230646 |
|
Type |
PeekNextGetMore |
|
ID |
0000000000000000 |
|
Total |
0 |
|
Queue |
0 |
|
Bytes |
12 |
|
Loc |
7252777665578 |
|
End |
7252781666374 |
|
HasMessage |
1 |
|
Version |
7252777665578 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588950096.127790 |
|
Type |
PeerDestroy |
|
ID |
0000000000000000 |
|
Error |
connection_failed |
|
ErrorDescription |
Network connection failed |
|
ErrorCode |
1026 |
|
SuppressedEventCount |
0 |
|
PeerAddr |
10.180.50.77:4518 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949824.773677 |
|
OriginalTime |
1588949824.583129 |
|
Type |
ProcessMetrics |
|
ID |
0000000000000000 |
|
Elapsed |
5.00001 |
|
CPUSeconds |
0.045374 |
|
MainThreadCPUSeconds |
0.044557 |
|
UptimeSeconds |
7.57623e+06 |
|
Memory |
7174557696 |
|
ResidentMemory |
6983024640 |
|
UnusedAllocatedMemory |
741200064 |
|
MbpsSent |
0.103321 |
|
MbpsReceived |
0.0856574 |
|
DiskTotalBytes |
1056755048448 |
|
DiskFreeBytes |
979928698880 |
|
DiskQueueDepth |
0 |
|
DiskIdleSeconds |
4.93201 |
|
DiskReads |
107 |
|
DiskWrites |
29 |
|
DiskReadsCount |
45720243 |
|
DiskWritesCount |
112056016 |
|
DiskWriteSectors |
1520 |
|
DiskReadSectors |
856 |
|
FileWrites |
200 |
|
FileReads |
107 |
|
CacheReadBytes |
81920 |
|
CacheFinds |
678 |
|
CacheWritesBlocked |
0 |
|
CacheReadsBlocked |
107 |
|
CachePageReadsMerged |
0 |
|
CacheWrites |
381 |
|
CacheReads |
127 |
|
CacheHits |
562 |
|
CacheMisses |
107 |
|
CacheEvictions |
0 |
|
ZoneID |
60062d710b037cb0bf737e2d7278790d |
|
MachineID |
60062d710b037cb0bf737e2d7278790d |
|
AIOSubmitCount |
118 |
|
AIOCollectCount |
126 |
|
AIOSubmitLag |
4.35922e-09 |
|
AIODiskStall |
1.19493e-05 |
|
CurrentConnections |
58 |
|
ConnectionsEstablished |
0 |
|
ConnectionsClosed |
0 |
|
ConnectionErrors |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949862.540402 |
|
Type |
ProcessTimeOffset |
|
ID |
0000000000000000 |
|
ProcessTime |
1588949862.540403 |
|
SystemTime |
1588949862.540403 |
|
OffsetFromSystemTime |
0.000000 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949824.773677 |
|
OriginalTime |
1581373597.325596 |
|
Type |
ProgramStart |
|
ID |
0000000000000000 |
|
RandomSeed |
2847564695 |
|
SourceVersion |
20566f2ff06a7e822b30e8cfd91090fbd863a393 |
|
Version |
6.2.15 |
|
PackageName |
6.2 |
|
FileSystem |
`` | |
DataFolder |
/mnt/fdb/storage/4500 |
|
WorkingDirectory |
/ |
|
ClusterFile |
/etc/foundationdb/fdb.cluster |
|
ConnectionString |
fdb_chaos_testing:[email protected]:451610.180.50.59:4516,10.180.50.77:4516,10.180.50.80:4516,10.180.50.179:4516 |
|
ActualTime |
1581373597 |
|
CommandLine |
/usr/sbin/fdbserver --cache_memory=5GiB --cluster_file=/etc/foundationdb/fdb.cluster --datadir=/mnt/fdb/storage/4500 --listen_address=public --locality_data_hall=3 --logdir=/var/log/foundationdb --logsize=50MiB --maxlogssize=800MiB --memory=13GiB --public_address=auto:4500 --trace_format=json |
|
BuggifyEnabled |
0 |
|
MemoryLimit |
13958643712 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588606950.607957 |
|
OriginalTime |
1586587210.082944 |
|
Type |
ProxyMetrics |
|
ID |
d8ea53f2da943a23 |
|
Elapsed |
5.00001 |
|
TxnStartIn |
1.2 3.05246 282 |
|
TxnStartOut |
1.2 3.05327 282 |
|
TxnStartBatch |
5.39999 8.84786 1084 |
|
TxnSystemPriorityStartIn |
0.399999 2.07089 61 |
|
TxnSystemPriorityStartOut |
0.399999 2.07146 61 |
|
TxnBatchPriorityStartIn |
0 0 2 |
|
TxnBatchPriorityStartOut |
0 0 2 |
|
TxnDefaultPriorityStartIn |
0.799999 2.49219 219 |
|
TxnDefaultPriorityStartOut |
0.799999 2.49272 219 |
|
TxnCommitIn |
0 0 10 |
|
TxnCommitVersionAssigned |
0 0 10 |
|
TxnCommitResolving |
0 0 10 |
|
TxnCommitResolved |
0 0 10 |
|
TxnCommitOut |
0 0 10 |
|
TxnCommitOutSuccess |
0 0 10 |
|
TxnConflicts |
0 0 0 |
|
CommitBatchIn |
0.2 0.209116 52 |
|
CommitBatchOut |
0.2 0.209867 52 |
|
MutationBytes |
0 0 7164 |
|
Mutations |
0 0 16 |
|
ConflictRanges |
0 0 38 |
|
KeyServerLocationRequests |
0.399999 1.93754 60 |
|
LastAssignedCommitVersion |
5214997671825 |
|
Version |
5214997671825 |
|
CommittedVersion |
5214997671825 |
|
CommitBatchesMemBytesCount |
0 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.459838 |
|
Type |
ProxyRatekeeperChanged |
|
ID |
ef3357ccedce9d99 |
|
RKID |
3ebe1a9f1f09a897 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.462527 |
|
Type |
ProxyReadyForReads |
|
ID |
ef3357ccedce9d99 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.459838 |
|
Type |
ProxyReadyForTxnStarts |
|
ID |
ef3357ccedce9d99 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.236639 |
|
Type |
ProxyReplies |
|
ID |
610cf5205c59d595 |
|
WorkerID |
8931592061f8bc6b |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.235072 |
|
Type |
RTSSComplete |
|
ID |
610cf5205c59d595 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.230646 |
|
Type |
RecSnapshotEnd |
|
ID |
610cf5205c59d595 |
|
NextKey |
\xff/tagLocalityList/\x00\x00 |
|
Nextlocation |
0.7252777665578 |
|
IsSnapshotEnd |
1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589306128.020088 |
|
Type |
RecruitStorageNotAvailable |
|
ID |
71f062b5aa2e75b9 |
|
Error |
no_more_servers |
|
ErrorDescription |
Not enough physical servers available |
|
ErrorCode |
1008 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032655.510371 |
|
OriginalTime |
1588624446.749546 |
|
Type |
RecruitedMasterWorker |
|
ID |
71f062b5aa2e75b9 |
|
Address |
10.180.50.77:4516 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620801.775621 |
|
Type |
RedundantConnection |
|
ID |
e78534b52842eb94 |
|
SuppressedEventCount |
1 |
|
FromAddr |
10.180.50.80:33774 |
|
CanonicalAddr |
10.180.50.80:4517 |
|
LocalAddr |
10.180.50.179:4516 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588947771.083477 |
|
Type |
RelocateShard |
|
ID |
47cedbda4d178447 |
|
BeginPair |
2e60eacc0d8324b0 |
|
KeyBegin |
\x15\x01\x02\x02\x05\x0d\x00\x00\x00\x00\x00\x00\x00+ |
|
KeyEnd |
\x15\x01\x02\x02\x05\x0d\x00\x00\x00\x00\x00\x00\x01J |
|
Priority |
950 |
|
RelocationID |
2e60eacc0d8324b0 |
|
SuppressedEventCount |
0 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588947771.102311 |
|
Type |
RelocateShardHasDestination |
|
ID |
47cedbda4d178447 |
|
PairId |
2e60eacc0d8324b0 |
|
DestinationTeam |
3f07c3c89688db6849c4e6e98ac35bc5753172b1ec946a9016a993c2270a8c91,ec81cb9448fdee9c84611a1757a148c0 |
|
ExtraIds |
[no items] |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588948041.077504 |
|
Type |
RelocateShardMergeMetrics |
|
ID |
47cedbda4d178447 |
|
OldKeys |
mako005282 - mako0054021 |
|
NewKeys |
mako005282 - mako0123825 |
|
EndingSize |
0 |
|
BatchedMerges |
79 |
|
LastLowBandwidthStartTime |
1.58889e+09 |
|
ShardCount |
79 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588947771.086890 |
|
Type |
RelocateShardStartSplitx100 |
|
ID |
47cedbda4d178447 |
|
Begin |
\x15\x06\x02\x02\x05\x0d\x00\x00\x00\x00\x00\x00\xfd\x99 |
|
End |
\x15\x06\x02\x02\x05\x0d\x00\x00\x00\x00\x00\x01\x00M |
|
MaxBytes |
35664020 |
|
MetricsBytes |
35927694 |
|
Bandwidth |
Low |
|
BytesPerKSec |
0 |
|
NumShards |
2 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
30 |
|
Time |
1588952239.804462 |
|
Type |
RelocateShardTooLong |
|
ID |
0000000000000000 |
|
Duration |
606.35 |
|
Dest |
1ddcb14a4183c13170c7fc8b0ce1f8be673a4fea98c33d79bd4a92cfcf3ff0ec,9e20cee080b698117a03822b46bc4bdb |
|
Src |
1ddcb14a4183c13170c7fc8b0ce1f8be669c1758985b5952094370cdae7fcbaf,e05471afd96a7c806a3bcd7d9173f08c |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588951378.208062 |
|
Type |
RelocateShard_FinishMoveKeysRetrying |
|
ID |
bb8989e8b32b1696 |
|
Error |
transaction_too_old |
|
ErrorDescription |
Transaction is too old to perform reads or be committed |
|
ErrorCode |
1007 |
|
KeyBegin |
mako1604058 |
|
KeyEnd |
mako1611403 |
|
IterationBegin |
mako1604058 |
|
IterationEnd |
mako1611403 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624414.664221 |
|
Type |
ResolverInit |
|
ID |
eb714dfc38eaa791 |
|
RecoveryCount |
56 |
|
Machine |
10.180.50.179:4517 |
|
LogGroup |
default |
|
Roles |
RV |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.236639 |
|
Type |
ResolverReplies |
|
ID |
610cf5205c59d595 |
|
WorkerID |
4e8dade3aca5e1cf |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624428.691057 |
|
Type |
ResolverTerminated |
|
ID |
eb714dfc38eaa791 |
|
Error |
worker_removed |
|
ErrorDescription |
Normal worker shut down |
|
ErrorCode |
1202 |
|
Machine |
10.180.50.179:4517 |
|
LogGroup |
default |
|
Roles |
RV |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589177956.122590 |
|
OriginalTime |
1589177956.048386 |
|
Type |
RkUpdate |
|
ID |
0000000000000000 |
|
TPSLimit |
6.7099e+07 |
|
Reason |
2 |
|
ReasonServerID |
07e2bd4103ad63a4 |
|
ReleasedTPS |
7.23594 |
|
ReleasedBatchTPS |
0.222651 |
|
TPSBasis |
7.23594 |
|
StorageServers |
36 |
|
Proxies |
3 |
|
TLogs |
6 |
|
WorstFreeSpaceStorageServer |
907272409498 |
|
WorstFreeSpaceTLog |
886403637658 |
|
WorstStorageServerQueue |
5995 |
|
LimitingStorageServerQueue |
5904 |
|
WorstTLogQueue |
1755 |
|
TotalDiskUsageBytes |
1537087093944 |
|
WorstStorageServerVersionLag |
0 |
|
LimitingStorageServerVersionLag |
0 |
|
WorstStorageServerDurabilityLag |
13965172 |
|
LimitingStorageServerDurabilityLag |
13951194 |
|
Machine |
10.180.50.80:4518 |
|
LogGroup |
default |
|
Roles |
RK |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589177956.122590 |
|
OriginalTime |
1589177956.048386 |
|
Type |
RkUpdateBatch |
|
ID |
0000000000000000 |
|
TPSLimit |
4.84604e+07 |
|
Reason |
2 |
|
ReasonServerID |
07e2bd4103ad63a4 |
|
ReleasedTPS |
7.23594 |
|
ReleasedBatchTPS |
0.222651 |
|
TPSBasis |
7.23594 |
|
StorageServers |
36 |
|
Proxies |
3 |
|
TLogs |
6 |
|
WorstFreeSpaceStorageServer |
907272409498 |
|
WorstFreeSpaceTLog |
886403637658 |
|
WorstStorageServerQueue |
5995 |
|
LimitingStorageServerQueue |
5904 |
|
WorstTLogQueue |
1755 |
|
TotalDiskUsageBytes |
1537087093944 |
|
WorstStorageServerVersionLag |
0 |
|
LimitingStorageServerVersionLag |
0 |
|
WorstStorageServerDurabilityLag |
13965172 |
|
LimitingStorageServerDurabilityLag |
13951194 |
|
Machine |
10.180.50.80:4518 |
|
LogGroup |
default |
|
Roles |
RK |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588967689.669706 |
|
Type |
AsyncFileKAIOOpen |
|
ID |
0000000000000000 |
|
Filename |
/mnt/fdb/transaction/4519/logqueue-V_3_LS_2-ebfd05340047e0023ebfd5976921797d-0.fdq |
|
Flags |
983046 |
|
Mode |
1408 |
|
Fd |
76 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624438.608650 |
|
Type |
SecondCommitBatch |
|
ID |
b2ca1c090d12d855 |
|
DebugID |
c005e9c6479e71db |
|
Machine |
10.180.50.179:4517 |
|
LogGroup |
default |
|
Roles |
MP |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588948086.506063 |
|
Type |
ShardServerFutureVersion1000x |
|
ID |
15e62ed9c0f0e5df |
|
Version |
7576974507598 |
|
MyVersion |
7576974506620 |
|
ServerID |
15e62ed9c0f0e5df |
|
Machine |
10.180.50.50:4503 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588951241.348125 |
|
Type |
SlowKAIOLaunch |
|
ID |
0000000000000000 |
|
IOSubmitTime |
0.016397 |
|
TruncateTime |
5.00679e-06 |
|
TruncateCount |
0 |
|
TruncateBytes |
0 |
|
LargestTruncate |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588970316.811875 |
|
Type |
SlowKAIOTruncate |
|
ID |
0000000000000000 |
|
TruncateTime |
0.0778663 |
|
TruncateBytes |
14774272 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588949830.494709 |
|
Type |
SlowSSLoopx100 |
|
ID |
739f9922184049d5 |
|
Elapsed |
0.0617683 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589222264.244907 |
|
Type |
Net2SlowTaskTrace |
|
ID |
0000000000000000 |
|
TraceTime |
1589222264.383268 |
|
Trace |
addr2line -e fdbserver.debug -p -C -f -i 0x7fbe95d375f0 0x69c3d0 0xf648ca 0xfa933e 0xfaa7f7 0xf6b111 0xf69469 0xfad5ec 0xf69a19 0xf6a2ab 0xfada46 0xfb6cac 0xf8602e 0xf865eb 0xf871bb 0xf87697 0x69f938 0xf64ef3 0xf650f1 0xf72e8b 0xf72ffc 0x69f938 0xdd1a79 0xb1be48 0xf9fb54 0xb52a51 0x19d8d88 0x663015 0x7fbe95472505 0x68f2f5 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589244944.324415 |
|
Type |
SomewhatSlowRunLoopBottom |
|
ID |
0000000000000000 |
|
Elapsed |
0.0427301 |
|
Machine |
10.180.50.179:4502 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588950070.505679 |
|
Type |
SomewhatSlowRunLoopTop |
|
ID |
0000000000000000 |
|
Elapsed |
0.0206292 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949825.432928 |
|
Type |
SpringCleaningMetrics |
|
ID |
739f9922184049d5 |
|
SpringCleaningCount |
7601254 |
|
LazyDeletePages |
10386661 |
|
VacuumedPages |
23618478 |
|
SpringCleaningTime |
8966.72 |
|
LazyDeleteTime |
4754.18 |
|
VacuumTime |
4199.48 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949824.773677 |
|
OriginalTime |
1588949822.486929 |
|
Type |
StorageMetrics |
|
ID |
739f9922184049d5 |
|
Elapsed |
5.00001 |
|
QueryQueue |
0 0 136693665 |
|
GetKeyQueries |
0 0 0 |
|
GetValueQueries |
0 0 124184553 |
|
GetRangeQueries |
0 0 12509112 |
|
FinishedQueries |
0 0 136693665 |
|
RowsQueried |
0 0 1635435284 |
|
BytesQueried |
0 0 979447889012 |
|
WatchQueries |
0 0 21 |
|
EmptyQueries |
0 0 11247314 |
|
BytesInput |
0 0 262719843608 |
|
BytesDurable |
0 0 262719843608 |
|
BytesFetched |
0 0 170977964670 |
|
MutationBytes |
0 0 132895506551 |
|
SampledBytesCleared |
0 0 203599898330 |
|
Mutations |
0 0 127402905 |
|
SetMutations |
0 0 88396221 |
|
ClearRangeMutations |
0 0 39006684 |
|
AtomicMutations |
0 0 0 |
|
UpdateBatches |
0.599999 1.90249 50512542 |
|
UpdateVersions |
0 0 10420748 |
|
Loops |
10.6 2.0473 264484588 |
|
FetchWaitingMS |
0 0 100233602 |
|
FetchWaitingCount |
0 0 91888 |
|
FetchExecutingMS |
0 0 574783160 |
|
FetchExecutingCount |
0 0 89939 |
|
ReadsRejected |
0 0 0 |
|
LastTLogVersion |
7578710932602 |
|
Version |
7578710932602 |
|
StorageVersion |
7578705932602 |
|
DurableVersion |
7578705932602 |
|
DesiredOldestVersion |
7578705932602 |
|
VersionLag |
633531 |
|
LocalRate |
100 |
|
FetchKeysFetchActive |
0 |
|
FetchKeysWaiting |
0 |
|
QueryQueueMax |
0 |
|
BytesStored |
3092723975 |
|
ActiveWatches |
0 |
|
WatchBytes |
0 |
|
KvstoreBytesUsed |
23042601712 |
|
KvstoreBytesFree |
979928698880 |
|
KvstoreBytesAvailable |
999224475648 |
|
KvstoreBytesTotal |
1056755048448 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589227661.943201 |
|
Type |
StorageServerNoLongerBehind |
|
ID |
4c48cfbd569beedc |
|
CursorVersion |
7856550574648 |
|
TLogVersion |
7856550574647 |
|
Machine |
10.180.50.179:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1585946133.285731 |
|
Type |
StorageServerRecruitment |
|
ID |
47cedbda4d178447 |
|
State |
Idle |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1589227527.665466 |
|
Type |
StorageServerUpdateLag |
|
ID |
4c48cfbd569beedc |
|
Version |
7856416307986 |
|
DurableVersion |
7856335382578 |
|
Machine |
10.180.50.179:4501 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.223162 |
|
Type |
TLogJoinedMe |
|
ID |
610cf5205c59d595 |
|
TLog |
70552e191c568533 |
|
Address |
10.180.50.179:4519 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.237857 |
|
Type |
TLogJoinedMeUnknown |
|
ID |
610cf5205c59d595 |
|
TLog |
42ea2e110cb30ee0 |
|
Address |
10.180.50.179:4519 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.222850 |
|
Type |
TLogLockStarted |
|
ID |
610cf5205c59d595 |
|
TLog |
701adf5893b59071 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.223495 |
|
Type |
TLogLocked |
|
ID |
610cf5205c59d595 |
|
TLog |
4b24114eba5b6c6c |
|
End |
7252781666373 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588964390.566463 |
|
OriginalTime |
1588964387.589247 |
|
Type |
TLogMetrics |
|
ID |
d168134a9dfed8e7 |
|
Elapsed |
5.00001 |
|
BytesInput |
199.999 367.144 358402462588 |
|
BytesDurable |
784.598 172.76 358402461588 |
|
Version |
7593276200217 |
|
QueueCommittedVersion |
7593276200217 |
|
PersistentDataVersion |
7593273726639 |
|
PersistentDataDurableVersion |
7593273726639 |
|
KnownCommittedVersion |
7593274051719 |
|
QueuePoppedVersion |
7593265945322 |
|
MinPoppedTagVersion |
7593265945322 |
|
MinPoppedTagLocality |
0 |
|
MinPoppedTagId |
1 |
|
SharedBytesInput |
1179255431786 |
|
SharedBytesDurable |
1179255430786 |
|
SharedOverheadBytesInput |
54142495985 |
|
SharedOverheadBytesDurable |
54142453570 |
|
KvstoreBytesUsed |
106118352 |
|
KvstoreBytesFree |
952354414592 |
|
KvstoreBytesAvailable |
952354414592 |
|
KvstoreBytesTotal |
1056755048448 |
|
QueueDiskBytesUsed |
50510909440 |
bytes used on disk for the files used by a particular instance |
QueueDiskBytesFree |
952354414592 |
free space on disk (e.g. as reported by the OS) |
QueueDiskBytesAvailable |
952354414592 |
amount available for storage engine use, to include free space on disk and any internal space within the storage engine’s files that’s not currently used but is reusable |
QueueDiskBytesTotal |
1056755048448 |
total size of the disk |
PeekMemoryReserved |
0 |
|
PeekMemoryRequestsStalled |
0 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624420.962159 |
|
Type |
TLogPeekAllAddingCurrent |
|
ID |
0000000000000000 |
|
Tag |
-1:1 |
|
Begin |
7252987965976 |
|
End |
7253101514298 |
|
BestLogs |
6562755249e03c5948296a5fb89a1da0 fd9ec644fe1a887e92f37eacca89e1e0, fcc74bd989146f9fec729f2986fa8591, 8e0b2719c40e23c8b080eb0f43aeea5c, 4e875de1e062f8b5c11510f911a2adc6 |
|
Machine |
10.180.50.179:4518 |
|
LogGroup |
default |
|
Roles |
MS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624420.962159 |
|
Type |
TLogPeekAllAddingOld |
|
ID |
0000000000000000 |
|
Tag |
-1:1 |
|
Begin |
7252987965976 |
|
End |
7253101514298 |
|
BestLogs |
60f915066a11b332df1bf98ae3f29b3d 33a3221be249eb338ef4540aed763ce7, c1d924e2f8406ff83301a25a1586bcb8, adf74fe090a23a9e2df7ea184f5fbf0f, db3ff73dcb39cfbe80903214d3ef896d |
|
LastBegin |
7252993510677 |
|
ThisBegin |
7252987965976 |
|
Machine |
10.180.50.179:4518 |
|
LogGroup |
default |
|
Roles |
MS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.225630 |
|
Type |
TLogPeekAllCurrentOnly |
|
ID |
0000000000000000 |
|
Tag |
-1:1 |
|
Begin |
7252777665578 |
|
End |
7252781666374 |
|
BestLogs |
701adf5893b590716c046170438ae770 4b24114eba5b6c6c161121f96c764bfa, 5af8954c2e85d21d50cb9472a392c008, f77128b5c4256d72d9968c7d8d523c38, d4d6e33e212b440a214c51d4feda4319 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.225630 |
|
Type |
TLogPeekTxs |
|
ID |
0000000000000000 |
|
Begin |
7252777665578 |
|
End |
7252781666374 |
|
LocalEnd |
7252779663229 |
|
PeekLocality |
-99 |
|
CanDiscardPopped |
1 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588970191.282965 |
|
Type |
TLogQueueCommitSlow |
|
ID |
ebfd05340047e002 |
|
LateProcessCount |
30 |
|
LoggingDelay |
1 |
|
Machine |
10.180.50.179:4519 |
|
LogGroup |
default |
|
Roles |
TL |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1585946140.458718 |
|
Type |
TeamCollectionInfo |
|
ID |
47cedbda4d178447 |
|
Primary |
1 |
|
AddedTeams |
0 |
|
TeamsToBuild |
0 |
|
CurrentTeams |
180 |
|
DesiredTeams |
180 |
|
MaxTeams |
900 |
|
StorageTeamSize |
3 |
|
CurrentMachineTeams |
8 |
|
CurrentHealthyMachineTeams |
8 |
|
DesiredMachineTeams |
30 |
|
MaxMachineTeams |
150 |
|
TotalHealthyMachines |
6 |
|
MinTeamsOnServer |
10 |
|
MaxTeamsOnServer |
17 |
|
MinMachineTeamsOnMachine |
4 |
|
MaxMachineTeamsOnMachine |
4 |
|
DoBuildTeams |
0 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032664.325156 |
|
Type |
TimeKeeperCommit |
|
ID |
8cd1b5993774ae29 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588935347.800644 |
|
OriginalTime |
1588935347.593266 |
|
Type |
TotalDataInFlight |
|
ID |
47cedbda4d178447 |
|
Primary |
1 |
|
TotalBytes |
0 |
|
UnhealthyServers |
0 |
|
ServerCount |
36 |
|
StorageTeamSize |
3 |
|
HighestPriority |
140 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |
|
TrackLatestType |
Rolled |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620842.930019 |
|
Type |
TransactionAttachID |
|
ID |
4ac2144aefd496c6 |
|
To |
edd8cd7acb37f07f |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1589032664.326260 |
|
Type |
TransactionCommit |
|
ID |
0000000000000000 |
|
BeginPair |
95aa1ffaebc10be3 |
|
Parent |
8cd1b5993774ae29 |
|
Machine |
10.180.50.59:4516 |
|
LogGroup |
default |
|
Roles |
CCCD |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588620806.471218 |
|
Type |
TransactionDebug |
|
ID |
9d42d60dd934fe08 |
|
Location |
MasterProxyServer.masterProxyServerCore.GetRawCommittedVersion |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMP |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588949825.792440 |
|
Type |
TransactionMetrics |
|
ID |
189fd7818f8615af |
|
Elapsed |
5.00001 |
|
Cluster |
`` | |
Internal |
1 |
|
ReadVersions |
0 0 0 |
|
LogicalUncachedReads |
0 0 2133637 |
|
PhysicalReadRequests |
0 0 2134174 |
|
CommittedMutations |
0 0 1 |
|
CommittedMutationBytes |
0 0 567 |
|
CommitStarted |
0 0 1 |
|
CommitCompleted |
0 0 1 |
|
TooOld |
0 0 0 |
|
FutureVersions |
0 0 0 |
|
NotCommitted |
0 0 0 |
|
MaybeCommitted |
0 0 0 |
|
ResourceConstrained |
0 0 0 |
|
ProcessBehind |
0 0 0 |
|
MeanLatency |
0 |
|
MedianLatency |
0 |
|
Latency90 |
0 |
|
Latency98 |
0 |
|
MaxLatency |
0 |
|
MeanRowReadLatency |
0 |
|
MedianRowReadLatency |
0 |
|
MaxRowReadLatency |
0 |
|
MeanGRVLatency |
0 |
|
MedianGRVLatency |
0 |
|
MaxGRVLatency |
0 |
|
MeanCommitLatency |
0 |
|
MedianCommitLatency |
0 |
|
MaxCommitLatency |
0 |
|
MeanMutationsPerCommit |
0 |
|
MedianMutationsPerCommit |
0 |
|
MaxMutationsPerCommit |
0 |
|
MeanBytesPerCommit |
0 |
|
MedianBytesPerCommit |
0 |
|
MaxBytesPerCommit |
0 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588624381.225630 |
|
Type |
UpdatedLocalityForDcId |
|
ID |
0000000000000000 |
|
DcId |
[not set] |
|
Locality0 |
-99 |
|
Locality1 |
-99 |
|
Version |
7252779663229 |
|
Machine |
10.180.50.179:4516 |
|
LogGroup |
default |
|
Roles |
CDMS |
Metric Name | Example | Description |
---|---|---|
Severity |
10 |
|
Time |
1588951217.218532 |
|
Type |
WaitForVersion1000x |
|
ID |
0000000000000000 |
|
Machine |
10.180.50.179:4500 |
|
LogGroup |
default |
|
Roles |
SS |
Metric Name | Example | Description |
---|---|---|
Severity |
20 |
|
Time |
1588948118.311610 |
|
Type |
WaitStorageMetricsPenalty |
|
ID |
0000000000000000 |
|
Keys |
mako09685116 - mako1626817 |
|
Limit |
100 |
|
JitteredSecondsOfPenitence |
15 |
|
Machine |
10.180.50.133:4516 |
|
LogGroup |
default |
|
Roles |
DD |