64 Collaborator |
Brandon Williams , Aleksey Yeschenko , Mick Semb Wever , Marcus Eriksson , Eric Evans , Benedict Elliott Smith , Sam Tunnicliffe , Yuki Morishita , Benjamin Lerer , Štefan Miklošovič , Alex Petrov , Berenguer Blasi , Ekaterina Dimitrova , David Capwell , Alexandre Dutra , Josh McKenzie , Andrés de la Peña , Robert Stupp , Caleb Rackliffe , Blake Eggleston , Dinesh Joshi , Francisco Guerrero , Stefan Podkowinski , Jacek Lewandowski , Jon Meredith , Chris Lohfink , Adam Holmberg , Jordan West , Aleksei Zotov , Doug Rohrer , Brad Schoening , Vinay Chella , Maxwell Guo , Saranya Krishnakumar , Michael Kjellman , Gianluca Righetto , Jyothsna Konisa , Deepak Vohra , Bernardo Botella , James Berragan , Maulin Vasavada , Arjun Ashok , Alexander Dejanovski , Yuriy Semchyshyn , jberragan , by , jkonisa , Shailaja Koppu , Raymond Welgosh , Francisco Guerrero Hernandez , reviewed by Stefan Miklosovic , Jyothsna konisa , Ben Manes , rwelgosh , reviewed by Yifan Cai , reviewed by Brad Schoening , Yuqi Gu , Yifan , Saranya , Marcus Eriksson, Jon Meredith , Francisco , Doug , Dinesh , Andres De la Pena |
134 Patch |
139 Review |
659558c980c67a80287ca7ccdfc8a70b1a56b7e2,
50273d98e4780b57da37400752eab69e65cd41bc,
933137fb14d4ebfe38d0d2c8e23b765aa1c89333,
473656c1d53edb998aa60d414221e397797de52b,
31b607c92a98de5eeb33cadfd973c5fc76656b07,
4526b3fcbde22d09065820286dd434d93ecc89ba,
84e10d0cad5e34ef50d6951ca33714dfde0caa53,
99d034a2245c44becb6a730c77ad51ab9340f3a7,
ffc4c89c3df7ad0ae73ebefdcb7e15a2790c0a52,
26dd119679605bf61ad3caa24a70509e5be5aac9,
8fc39685fbb6c963cf0d86fd458069ab80bbcece,
c48906394e01460382f4070ecc34f6f9754fc567,
b83da7d978f1bae5f289f1561dd080d48dc4505b,
8327744c1aca0d9471cd8a01a1c7d79365363816,
4094e9ce2c4e7e367a319fe4c6d947340888d819,
3a6f6907314670fdb2b316db8f08ffd85da88851,
d9460a04daee5fa97639abf2b6e28ff9b29cf636,
4aab2c79b9539e8d8cfe90e4dd700b8da2f9e8a0,
97bfde01f805b4c3abd52870e0338440093f5893,
79bbb7dac5ad62f3bc725028ca7ea2be351d1999,
42025fd73a19597220192d71585ccb8baee7ed2a,
8acbbe042b236c6948845ecd7af093c6f0fa3e4b,
fd69375af0d31dccf4e14404ac58c9ced0f64dd9,
845077fbc8d7102f303372eb8cb3299a155a436e,
0fd8f0a52fbd69c47d073373abfe7d2437bbd9ca,
b915688ea878aaa284f5cedeb799c5f797c4d824,
a1285ac92ded45ab6e9f6c7c98917daf14a4a320,
c591978f4d265e42d0132418005ba63a99278c75,
425f750a6ffcc65340b62692548e38855d76040f,
f32475a839e01e4eea3989871d293d70e8a360d7,
d6f18106f09e7b1a58744c9bd1ee35054a43d68a,
0373037a0db7e13548b0f302cad9414f00e58991,
d12cb030690a2673a88d043a265382bdbc7f0009,
c6ea8fabf9f726a58b76ff6e41d21b650941f880,
46ee939b957528185dc6bbd3028c1d6e695163e7,
fde640fe52704836ec21fedd62cae21290e099ec,
e5ab8c1951384b9ddf0df9f1d4d49b4c9dfc188f,
fee7a10823da1e29bd0e6504fea9679389180c9e,
ce270081bc0bc8ffa0a7e1e5c04f30b5c1875a84,
f15c6b8c06c9588bd96615fabfc36b11857cd4bb,
1728da30e4e7858d30178ef74350af3e690adf0c,
83e1e9e45193322f18f57aa7cc4ad31d9d5a152d,
7f258f1ef6d79c908c4d846d8616d8f5554282ea,
0e65e85861b84e87691d576ba80c5c3b3b7020d5,
d51c18f807c5c88ef44114341592214747487645,
f3198c45067f753444ffafbc83b6aa563cb392ac,
9251b8116ff89b528b6b9eaa43d4dc2d1bc0bbaf,
43c19878e38fbe260f9e6143aa43836e85cf2f44,
da95e4b3fb36294b5117846b8fbb8cdc01e427d5,
753b40eb0f570fc88b5211b9bcea04761a240071,
e08053b77cac4ec91fd398d7bad65bba1394f45f,
abeaa3ea5ef99691cc1b29787cfcd573a90e34fb,
d3b3dcbb353de97220a11f55391babf149410905,
0e3a90698a94772e57df39e7461efe6b7e09d678,
1fbd3297a9c8303ca7aa2ff30d182e5ca568de4c,
9abe2127dde7ea317928b37b8b5c662e787b2192,
5459c121d5d688ec1571dd72550bd6faaeb087eb,
647bdd6a11970f80666d7f20b53af76fbda4ff14,
bef152d63018c521a84874e3eafe55663511f5b2,
7bf5be87e1d37e9f9302f15b1b79a48b0082b521,
5db813c579696bd51f5327a92fef59588367e4d4,
f5bc21c40ccd4bc2b9bc118ec5888bad3cc15b16,
5b26885187d0aa8bc09cd1b1f4b6be81aebb40c7,
9b37af1398b55c183d830c8a4c10e0c2f33b43d3,
8a89c816e1b78e7a998d9006c3f7d5749c04b053,
238fe175af9f4b429d170ffd165cb990e174ee95,
8d88fba753db9a464bb0b562b87bd0d5b6271c69,
04b0d095c233f1b2f918446852c326caaf0125a2,
743a623605dbb1c265fd93563b436b956a498e70,
77bb1ca3417019a077c88c4933337eaf42e069ea,
7b8b607c848f107d16c1523d06246a471a55d410,
f3bcbba3dcd81b640711baa35a76a2d949ce6c5e,
213274b82fb4dda98a028de7cd131371b4c1d826,
f07e248d0ce8303a06daf93b462190ef7be7304d,
4601e28529996a3447e74093cc6cc35879143031,
4fbdb880a4369700aed51ce885a5b349d28c0397,
55a9efee30555d3645680c6524043a6c9bc1194b,
f045207b623fde27b84d0f717b3075ea20726502,
00fba404a00282ec2117290fdf45658387a2b016,
10034a4e9c9913127e28546f2263f2ce818f6a37,
4341f1e00f92b9da132e13acd4eb4760f1d89e3f,
7ecef7c3c2e2137d4cb045ccf429703cacf96370,
385ab5444884db09e7e018e24416a13198ea5139,
ec887e063e5299d30c761c9d05ea4dc9e5ab8a51,
55866caa2c4601b1d59a8532a97310a9e819931f,
22b6df435936bdf0a616cf811c9a84821d4eae3c,
686d9e8699bd0a56857e24523d883120023b9841,
9d61f3568b7961de3f95e567d461e5103441a161,
ca1de4cc2e594266c1fca9f83cfcac38d9a21c45,
d19a1b9a112614d378b42bb136cc84ae9cc6213a,
835b0f72b5e8a6fd5c9ac29b97a47942b97740b2,
2dcbca66a9f6a1e6b7cf4bd2c4f1368cc2ede86a,
c72f2179143e7e031f247d3e8385a29c5e64c1c3,
ee454741363f3f693726af242c5ec37ad1480d60,
1b6052a8c3b04fca48f82e887c12bcfb2bae0bdb,
acea8725c0ff62cfcf538515f4b94a97e23414b4,
6deef0107ae4cf1c18ca67bfbfd225619bf54ce1,
5597b7ec3c87d2dae202bb4f4ac583aeff902231,
109d9abf8d2c7816378ed078b6151baa50fc4ab8,
6e8a62845acd211bb8fa460ef874b477245f395e,
6224c618496e06390b28276812844aabe0182e64,
098ae42efd555bfa99211ae7da6b9452e77f7257,
38cdacb2e7418e2aefbcffb1754dcd324c46028d,
c5669b5e84e33109c407944563e8c466b5a14717,
083772f05163fb224157b2fb07ff3c1df7a240b8,
29485e077526a1730e3ba0cb3ca268cfd060462a,
96f43fe0c831cd1317c90ecbd7463737fa439aec,
e51716ee724cf4950df67eba0393b3f798b7dc00,
dd536f2e70118cd5d0c319f5be3e54e3d50eb288,
a3ca0897c2190c2c18992ca2b7e5255318ff3eba,
cfef011d9745b43226dc1936f6e1778d3379b53d,
6556d251bdddfbef3935da760bcda2b2387a4391,
4624a17098e055e0abf9a6025451d4352cb9c147,
ff9ac41b4695c1df59f5293f69e0d3a1ce0da9f4,
4fb1e7f47d640353cd57f7a3035c70099049b29c,
f123406e458c0112145f37dcd3f8c20ba47c949d,
8655ca54a5d0749fccb2ad6a06ec230e8b0de24e,
cfe293dadcf7a1d4491591cfd39fc410a8fa52ba,
555e8494d3ca27a7b35aebabb1f669eede20cc53,
d75a6bae5abbf80810012a181644f240141014d5,
dbbd211cd420eb185d0579f16f5d46abc7bafeb4,
e168011c40de2ca48d138514640838067e61feea,
798182a6fda562538c2f44e4f3f92a7cb68cd81c,
2a693d721182c1d354ff1323b1324fe06ac03f36,
466e7bf5e160d1667c12ea1de1b79ba27670aba4,
aea798dc7e517af520a403d4d86f3bc6bed65092,
a13532272051d4e4608f92d53bdd997103e8ea19,
c3e8803b3331bc7ef81797ac52a8417524f67edc,
e0ae9d7484e242f6af495aac2cb4d8dc121fba89,
b5ba5fad4df490d1b7d47889361db910589409b8,
bbfca46129992e83055ba9b0b4f836871eef0990,
680cc9395c55a88217f2de975f62ad588e8c95d5,
88faba42e5cb3f1384c92024a9c3608135d76218,
1633cd9c6c3d88d5c66825fab76a369266509f7e |
e8891be04a681fbd87e2bec2c07ed22e55725a04,
ca729f6fad70eb07dcf590274e8664b865af2d42,
26c374da4f03e4a6b64e414805cd92f3eb0a36c6,
eb30005251cd8c10732ecab8365ebaa45f5fcbde,
6ffa43f68b8d10ca84d4a00bf81269527b4e14df,
9c796dfb272daa3ce57a2dc5cbeadd9273e1ac72,
933137fb14d4ebfe38d0d2c8e23b765aa1c89333,
f078c02cb58bddd735490b07548f7352f0eb09aa,
75194201f1f06d120f246f6fad025ca5f672943d,
acd815898d1e5669e7d8ee2469808f6e8a14a7df,
4a555f47ee943ce9fd70862cc8127d707e3507a2,
ccada788c47882bfb10d7cb86f7f39c9865428f2,
145dbd1f0875ae4c54392125e09ed05153c0dd8f,
36e16ee3c911c710129fcf3a69595038c3dbd385,
ca75ffe4d09a3e7b26a56345c0bdacaa284eaab7,
6748b8b7ead6fff7820045ebbfe23f6c050f2efe,
d0b9532f2b87a17a0508d0637556f2f3e8d0fd94,
83c169ec9e36324f27bf562951362f4a03c3c688,
09b282d1fdd7d6d62542137003011d144c0227be,
c8c8635a4c902ef051a46845919a5430f8d71e3f,
9a4a67782311e4a6ffe3aad0516c33016f187c54,
a250126f0f277b43a18cb665ccd02a105271bc33,
7119cf8be1c32fa6ac015456965ae27a17cd828c,
a9725b681b948f2122f3d48b96a5c4e7403d2c39,
557b8e9982ad0964191abde810ef5c77a536f70a,
ed3901823a5fe9f8838d8b592a1b7703b12e810b,
013acc641c5d487b07be5c082af1e85d26bd127f,
545809616c92a91e4c39d1eedfa65800f25a2a93,
1ad8bf67a9c82cbb5ff38e5cf785f9fe2516d009,
2fe1c304835ef39093b70cbb89107383be3c3ee9,
f1c1694e4329a1f6dba8b978edaed4312e235116,
61be4d836213f708d9a29e59b9ef1df0bebef29a,
fd5e2ad74eff61e189c2abe87797c2690a45220c,
543608ba39d5803b963d14821abe193ff0796b4f,
945a4fc23ac1f60b8380be3b60aef89caf3daba2,
0dc5a289e8dd586150253d951e6e229480c0ffc8,
2043cb9fb6b25ff34afb90467b9476a09acc3933,
a41cdd64b217c451b5576abe2f455eaa7ec1f322,
33fd2dc817f56116af0b4b7d2cd5365ce52948d9,
77dde2a3c4b40da3d820d4852c572338acbf6dc9,
b29e1037e4da75dfd2a30ad10f8008b24941e37f,
185893256f10c14207bffe49ae733fb1a970aec5,
2fba5c80ce7bf71d04c62043ffa1088b9e832d83,
5f23e6d766a18782db82d955bf380239990d2c84,
e6946e7ddb76bd72109995664289e988b0e9818c,
803a23b2f1b24701c22e7fadaf98c2c9a9491d35,
c52f50f046fd1a8e1c7f17d9a1952e2bd92aeb4c,
1185593907ba6597c29da1dfb5e6edb121b358c5,
3edacd632c2f2c360ca1d931c6b4f6fd7326511f,
efa25fc8d10bbfcefe14fc6f2a623b6a8b73b5cd,
d421e82ee0ffd66d3f382bfbe0b69b7b275edce3,
1ca5769c9adfd2e8bad3b7eea83112ad14cb05b2,
865b67b21d326728936f0b80681129c73a2e374a,
838bd8cae6d125cfb055dbf6bfc5a2d4e13cdfcb,
b74d7370cc89fa899f47f50c825ddaed2dd05c3f,
7cf35988f298a8ae4ce7ebab2c69e5e7abc28958,
d3e48e4e00b1e8bd45e0694c5a8d9a0e941fa985,
8333d0b0890f6299fac1fd219f58aee0c62cbf9c,
12b610246bc42dc6af33abfe0885b2f989fc2c73,
f02e53568dbc193b7ac75cc19b0a7751d5514b95,
786cb11e182c6cb47f36d11d68e52c198d31ba76,
876ac8c6111697c8105899889bf794438110bc74,
88b18a603c0affa05679b11c2e7d91324fa8f719,
beee6b441c71895ca7b2833631933a6a55b516c2,
4d1d024136dd5f4b3775a0666b0c30a65641d57f,
7e78059b38f367d1c98bb11df6ba78633c1516f0,
db5832120f0fe9ad799ca0b5b16bbcf6f7404580,
5be83b6a72695253c552535d2b826209f144cc63,
521a6e2aa9f8a4bc95dd13e768ec6de33cf6fa15,
4564e102684dc5f66ec73de1fc836f97a1fa33c9,
0eb8cec5c75a7dc9503f59505bed67da65c49503,
63b172e137e0306aefd84f373963d8014c5a5efa,
ba63fa3c951cb5c18d0fa4f9483577c6e18389c4,
8b25cd58bfa646db9e6c24c51896950da02945db,
fdcd0dff216d9e1ad242be1a7d5be3ef67044ac3,
05c70abc6350f87ed6cafbc66de8b0e46efe8e49,
b33a11ca4219b2317d35e48e1fb6f8a76b697e3e,
75d14b629da9360a8f2acd0153938b505c7c0714,
b2f2c70e26a32253b0e58ee197c2f8abf01dd449,
563592801aad00e002f0a162d5e2625a4b0e8723,
964051b04767689a1a3a84493c0b4afe77d6edfc,
1b71196a036b4f33d1ef53418bd21ac4b241399e,
672af9b56f1729c6511a2279923eb435df4b7b9b,
38cdacb2e7418e2aefbcffb1754dcd324c46028d,
e51716ee724cf4950df67eba0393b3f798b7dc00,
67281b31010791fa7f0d02dd0f776862e15846d3,
3bcc5297bd3115ff5949a9295eed6a9ad03fd096,
9f0f475ba87df6c631029748bfafb4169bfa6465,
3023a204c8ef16f886bd3dc219f7534b7edbaf2a,
bac08796181979afef4cc518789a380edef500f0,
84d84fe36b0d6e250c3d221c28c40b6925e4c222,
458a3630f882ae2b2a9cee272cf85ca7ff42f5cd,
a242b352c28947427a9bfc30295a487017439fd9,
b31a451873eb411788a6d94c3cacf881f5c3cb86,
8dce35f1cb3c204be669548ee286055b12e67fe9,
45de9e08e7dc177f1f3273456b62af7ee0f5dbdb,
f4014c06d7668541010d59cc932970e9ebfc36f5,
86420f9d52991fb148b322031df55494669532d3,
690101840d4d8f9c656bb0ca114f6619af80e1cf,
47fdb6448b6956249790d5dc7bb76b699d35c079,
cbbf33d001b6f953be5654f00d7dfb54011a7619,
295358095db80ced4b8f54f603f7bd9833a8f175,
c5d6dfd1bc9b682d704d28f77807ba72317b1944,
c00c454d698e5a29caf58e61ed52ab48d08fd7fe,
d28442ae712c1597052493aa3d2353a2de2495c2,
164243e78f1557a34bc699ebc716b532781d6422,
6ce33604bbd9acbee092ab3c4f7f11c0d434f730,
cf6de14d5b96ea173d6a1b2dad9bb64d563df06c,
46c35d0ef2efb66512133a7913df9936b0a80dc8,
dc0e79b9c483562ec0920d69e886715eb329c426,
d61e44f78fa4ba5ec395e1e39c507d666fddefd1,
fc08d45b283e701aa6d558e99cd18318394b0de7,
047d13806078bafdc3954273b0e240dbbb976bd4,
8c20b452dd0728a6fad6d276a7be9fa1b9274495,
d949d8c2b9813c3e8429ece34c364a356bd7d6eb,
fa6df8e2c09ad3d27bfe8c0ce016c839094630f6,
e8fb77f4813b469d73d39c84acf1e1fe7a40702b,
e82fceaecfe5ea04ac3ddff92be5a6a41456333c,
550bdfa1c6082537e2cfb93449128a61dbe3a1fb,
0aaf5659028dd874c8d666c636f11eae63c429e6,
672d66a64a21e23c4d81c089b426360c2bb708b7,
c7c3bbca2c7cb415b39689e924fa2357c239f043,
457b36bcb3c8a865cca83ca6c402246798113ab4,
c73c76498b0c2b36705025de6b0b2a7bb38e758b,
8771581b255e5728a16aea84430506d6f156a589,
deebdf97ad01f23550d7d3b42d98c7bf111e2f95,
f24951ab6ea2b1e9af4013b030675c70d31adb90,
014db08a79f00ef0d94e6855779e398c9dc689c1,
82b3c0a79c9322142738a4ec2ff7d4d4c0be2370,
6f8f404535d4cff9272091f669f985ce11cee7d2,
69766bca399cc779e0f2f8e859e39f7e29a17b7a,
912fbb47fddc07afcf56f5de97e813593bfb890e,
ee1c83722bfb1155bef762cdfb2c86034857f2d0,
bd0b41fb82134844a15fbb43126424d96706d08e,
9523a38b3f1b5bc4313e2949896ddc1fff58afbe,
f0fae2deeee20df15ac1105af2163af2a7e7953d,
b87b0edd310d1ef93c507bbbb1ae51e1b0b319c6,
7764214d1fb44fb6139a622f403bb05610e8f7b1,
1633cd9c6c3d88d5c66825fab76a369266509f7e |
e51716ee724cf4950df67eba0393b3f798b7dc00 | Author: jberragan <jberragan@gmail.com>
| 2024-12-06 16:09:12-08:00
CASSANDRA-19962: CEP-44 Kafka integration for Cassandra CDC using Sidecar (#87)
This is the initial commit for CEP-44 to introduce a standalone CDC module into the Analytics project. This module provides the foundation for CDC in the Apache Cassandra Sidecar.
This module provides:
- a standalone Cdc class as the entrypoint for initializing CDC.
- pluggable interfaces for: listing and reading commit log segments for a token range, persisting and reading CDC state, providing the Cassandra table schema, optionally reading values from Cassandra.
- read and deserialize commit log mutations.
- reconcile and de-duplicate mutations across replicas.
- serialize CDC state into a binary object for persistence.
- a layer for converting Cassandra mutations into a standard consumable format.
Patch by James Berragan, Jyothsna Konisa, Yifan Cai; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-19962
Co-authored-by: James Berragan <jberragan@apple.com>
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Jyothsna Konisa <jkonisa@apple.com>
238fe175af9f4b429d170ffd165cb990e174ee95 | Author: Yifan Cai <ycai@apache.org>
| 2024-12-04 10:53:40-08:00
CASSSIDECAR-169: Update asf.yaml to enable autolink after project renaming (#154)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSSIDECAR-169
8d88fba753db9a464bb0b562b87bd0d5b6271c69 | Author: Jyothsna konisa <jkonisa@apple.com>
| 2024-12-02 15:28:18-08:00
CASSSIDECAR-158: Adding APIs required for CDC (#147)
Patch by Jyothsna Konisa, Saranya Krishnakumar, Yifan Cai; Reviewed by Bernardo Botella, Francisco Guerrero, James Berragan, Yifan Cai for CASSSIDECAR-158
Co-authored-by: Jyothsna Konisa <jkonisa@apple.com>
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
Co-authored-by: Yifan Cai <ycai@apache.org>
04b0d095c233f1b2f918446852c326caaf0125a2 | Author: Yifan Cai <ycai@apache.org>
| 2024-11-20 15:21:34-08:00
CASSSIDECAR-165: Add metric to report consistency check duration (#150)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSSIDECAR-165
dd536f2e70118cd5d0c319f5be3e54e3d50eb288 | Author: Yifan Cai <ycai@apache.org>
| 2024-11-15 15:31:14-08:00
CASSANDRA-20066: Expose detailed bulk write failure message for better insight (#92)
Patch by Yifan Cai; Reviewed by Doug Rohrer, Francisco Guerrero for CASSANDRA-20066
743a623605dbb1c265fd93563b436b956a498e70 | Author: Yifan Cai <ycai@apache.org>
| 2024-11-06 12:57:53-08:00
CASSANDRASC-157: Update CircleCI config build with java doclint (#146)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-157
a3ca0897c2190c2c18992ca2b7e5255318ff3eba | Author: Yifan Cai <ycai@apache.org>
| 2024-11-05 14:22:45-08:00
CASSANDRA-19994: Add dataTransferApi and TwoPhaseImportCoordinator for coordinated write (#91)
Patch by Yifan Cai; Reviewed by Doug Rohrer, Francisco Guerrero for CASSANDRA-19994
77bb1ca3417019a077c88c4933337eaf42e069ea | Author: Yifan Cai <ycai@apache.org>
| 2024-10-31 10:05:33-07:00
CASSANDRASC-155: Fix dangling restore jobs (#143)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-155
7b8b607c848f107d16c1523d06246a471a55d410 | Author: Yifan Cai <ycai@apache.org>
| 2024-10-28 18:51:30-07:00
CASSANDRASC-154: Remove duplicated AbstractSchema (#142)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-154
f3bcbba3dcd81b640711baa35a76a2d949ce6c5e | Author: Yifan Cai <ycai@apache.org>
| 2024-10-28 11:17:38-07:00
CASSANDRASC-153: Fix RestoreRangeTask not terminating (#141)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-153
213274b82fb4dda98a028de7cd131371b4c1d826 | Author: Yifan Cai <ycai@apache.org>
| 2024-10-24 16:34:10-07:00
CASSANDRASC-152: Fix restore job of coordinated write not proceeding (#140)
- Extends the restore_job table to persist the total slice_count in order to known the complete data to restore
- Allows external to trigger restore job discover to be more responsive
- Fix restore job & range status update
- Fix two phase import implementation in RestoreProcessor
- Expose overall restore progress status in API response
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-152
3bcc5297bd3115ff5949a9295eed6a9ad03fd096 | Author: jberragan <jberragan@gmail.com>
| 2024-10-17 10:02:25-07:00
CASSANDRA-19980: Remove SparkSQL dependency from CassandraBridge so that it can be used independent from Spark (#88)
Patch by James Berragan; Reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19980
f123406e458c0112145f37dcd3f8c20ba47c949d | Author: Yifan Cai <ycai@apache.org>
| 2024-09-11 21:03:47-07:00
CASSANDRA-19909: Add writer options COORDINATED_WRITE_CONFIG to define coordinated write to multiple Cassandra clusters (#79)
The option specifies the configuration (in JSON) for coordinated write.
See org.apache.cassandra.spark.bulkwriter.coordinatedwrite.CoordinatedWriteConf.
When the option is present, SIDECAR_CONTACT_POINTS, SIDECAR_INSTANCES and LOCAL_DC are ignored if they are present.
Patch by Yifan Cai; Reviewed by Doug Rohrer, Francisco Guerrero for CASSANDRA-19909
9f0f475ba87df6c631029748bfafb4169bfa6465 | Author: Arjun Ashok <arjun_ashok@apple.com>
| 2024-09-05 13:24:01-07:00
CASSANDRA-19873: Removes checks for blocked instances from bulk-write path (#76)
Patch by Arjun Ashok; Reviewed by Yifan Cai, Francisco Guerrero for CASSANDRA-19873
cfe293dadcf7a1d4491591cfd39fc410a8fa52ba | Author: Yifan Cai <ycai@apache.org>
| 2024-08-30 11:08:00-07:00
CASSANDRA-19842: Consistency level check incorrectly passes when majority of the replica set is unavailable for write (#75)
Patch by Yifan Cai; Reviewed by Doug Rohrer, Francisco Guerrero for CASSANDRA-19842
f07e248d0ce8303a06daf93b462190ef7be7304d | Author: Yifan Cai <ycai@apache.org>
| 2024-08-23 15:50:40-07:00
CASSANDRASC-143: Enable github squash in asf.yaml
Patch by Yifan Cai; Reviewed by Francisco Guerrero, Stefan Miklosovic for CASSANDRASC-143
555e8494d3ca27a7b35aebabb1f669eede20cc53 | Author: Yifan Cai <ycai@apache.org>
| 2024-08-20 17:33:35-07:00
CASSANDRA-19836: Fix NPE when writing UDT values (#74)
When UDT field values are set to null, the bulk writer throws NPE
Patch by Yifan Cai; Reviewed by Dinesh Joshi, Doug Rohrer for CASSANDRA-19836
d75a6bae5abbf80810012a181644f240141014d5 | Author: Yifan Cai <ycai@apache.org>
| 2024-08-14 13:10:15-07:00
CASSANDRA-19827: Add job_timeout_seconds writer option (#73)
Option to specify the timeout in seconds for bulk write jobs. By default, it is disabled.
When `JOB_TIMEOUT_SECONDS` is specified, a job exceeding the timeout is:
- successful when the desired consistency level is met
- a failure otherwise
Patch by Yifan Cai; Reviewed by Dinesh Joshi, Doug Rohrer for CASSANDRA-19827
4601e28529996a3447e74093cc6cc35879143031 | Author: Yifan Cai <ycai@apache.org>
| 2024-08-13 23:29:44-07:00
CASSANDRASC-142: Improve S3 download throttling with range-GetObject
The patch adds a few changes according to the best practice for AWS SDK S3 client, including:
- download throttling with range-GetObject
- configure apiCallTimeout
- use OpenSSL for S3 client
Patch by Yifan Cai; Reviewed by Doug Rohrer, Saranya Krishnakumar for CASSANDRASC-142
e168011c40de2ca48d138514640838067e61feea | Author: Yifan Cai <ycai@apache.org>
| 2024-08-07 17:02:38-07:00
CASSANDRA-19806: Stream sstable eagerly when bulk writing to reclaim local disk space sooner (#69)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRA-19806
3023a204c8ef16f886bd3dc219f7534b7edbaf2a | Author: jberragan <jberragan@gmail.com>
| 2024-08-03 07:51:52+01:00
CASSANDRA-19807: Improve the core bulk reader test system to match actual and expected rows by concatenating the partition keys with the serialized hex string instead of utf-8 string (#70)
Patch by James Berragan; Reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19807
84d84fe36b0d6e250c3d221c28c40b6925e4c222 | Author: jberragan <jberragan@gmail.com>
| 2024-07-22 13:38:28-07:00
CASSANDRA-19791: Remove other uses of Apache Commons lang for hashcode, equality and random string generation (#67)
Patch by James Berragan; Reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19791
4fbdb880a4369700aed51ce885a5b349d28c0397 | Author: Yifan Cai <ycai@apache.org>
| 2024-07-18 14:46:49-07:00
CASSANDRASC-132: Add restore job progress endpoint and consistency check on restore ranges (#123)
The patch adds a new web API to retrieve the restore job progress that is computed using the current Cassandra cluster topology and the consistency level required for the restore job.
Patch by Yifan Cai; Reviewed by Bernardo Botella, Francisco Guerrero for CASSANDRASC-132
458a3630f882ae2b2a9cee272cf85ca7ff42f5cd | Author: jberragan <jberragan@gmail.com>
| 2024-07-17 14:29:21-07:00
CASSANDRA-19778: Split out BufferingInputStream stats into separate i… (#66)
Split BufferingInputStream stats into separate interface so class level generics are not required for the Stats interface
Patch by James Berragan; Reviewed by Bernardo Botella, Francisco Guerrero, Yifan Cai for CASSANDRA-19778
798182a6fda562538c2f44e4f3f92a7cb68cd81c | Author: Yifan Cai <ycai@apache.org>
| 2024-07-16 21:44:18-07:00
CASSANDRA-19772: Deprecate option SIDECAR_INSTANCES and replace with SIDECAR_CONTACT_POINTS (#63)
This patch introduces a new option SIDECAR_CONTACT_POINTS for both bulk writer and reader. The option name better describes the purpose, which is to specify the initial contact points to discover the cluster topology. The existing option SIDECAR_INSTANCES are used for the same purpose and it is now deprecated.
In addition, it allows including the port value in the addresses when defining SIDECAR_CONTACT_POINTS
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRA-19772
2a693d721182c1d354ff1323b1324fe06ac03f36 | Author: Yifan Cai <ycai@apache.org>
| 2024-07-16 15:20:52-07:00
CASSANDRA-19774: Bump Cassandra Sidecar version (#65)
Update Cassandra Sidecar commit sha: 55a9efee30555d3645680c6524043a6c9bc1194b
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRA-19774
55a9efee30555d3645680c6524043a6c9bc1194b | Author: Yifan Cai <ycai@apache.org>
| 2024-07-15 21:05:12-07:00
CASSANDRASC-139: Upgrade asciidoctor plugin to version 3.3.2 (#129)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-139
a242b352c28947427a9bfc30295a487017439fd9 | Author: jberragan <jberragan@gmail.com>
| 2024-07-12 14:57:38-07:00
CASSANDRA-19748: Refactoring to introduce new cassandra-analytics-common module with minimal dependencies (#62)
- Add new module cassandra-analytics-common with no dependencies on Spark or Cassandra and minimal standard dependencies (Guava, Jackson, Commons Lang Kryo etc)
- Move standalone classes to cassandra-analytics-common module.
Some additional refactoring and clean up:
- Rename SSTableInputStream -> BufferingInputStream
- Rename SSTableSource -> CassandraFileSource
- Introduce CassandraFile interface to be the implementing class for SSTable and CommitLog.
- Generalize IStats to work across different CassandraFile types
- Rename methods in StreamScanner to make the API clearer.
- Move ComplexTypeBuffer, ListBuffer, MapBuffer, SetBuffer, UdtBuffer to standalone classes
- Delete unused classes RangeTombstone, ReplciaSet and CollectionElement.
- Remove commons lang as a dependency
- Rename Rid to RowData
Patch by James Berragan; Reviewed by Bernardo Botella, Dinesh Joshi, Francisco Guerrero, Yifan Cai, Yuriy Semchyshyn for CASSANDRA-19748
f045207b623fde27b84d0f717b3075ea20726502 | Author: Yifan Cai <ycai@apache.org>
| 2024-07-01 12:48:14-07:00
CASSANDRASC-137: Bump AWS SDK version to 2.26.12 (#128)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-137
00fba404a00282ec2117290fdf45658387a2b016 | Author: Yifan Cai <ycai@apache.org>
| 2024-07-01 12:47:22-07:00
CASSANDRASC-136: Add delay between reporting of the same slow task (#127)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-136
b31a451873eb411788a6d94c3cacf881f5c3cb86 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-06-27 15:51:15-07:00
CASSANDRA-19727: Bulk writer fails validation stage when writing to a cluster using RandomPartitioner (#61)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19727
8dce35f1cb3c204be669548ee286055b12e67fe9 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-06-20 14:24:21-07:00
CASSANDRA-19716: Invalid mapping when timestamp is used as a partition key during bulk writes (#60)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19716
ca729f6fad70eb07dcf590274e8664b865af2d42 | Author: Bernardo Botella <contacto@bernardobotella.com>
| 2024-06-07 11:44:31-07:00
CASSANDRA-19685 - Add auto_hints_cleanup_enabled to web documentation
patch by Bernardo Botella; reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19685
10034a4e9c9913127e28546f2263f2ce818f6a37 | Author: Yifan Cai <ycai@apache.org>
| 2024-06-04 14:54:13-07:00
CASSANDRASC-134: Detect out of range data and cleanup using nodetool (#125)
The patch adds a step to check the data ownership before importing sstable. When fully out of range sstables are found, the sstables are removed before importing. When partially out of range sstables are found, running nodetool cleanup is requested on job completion, including both success and failure cases.
Patch by Yifan Cai; Reviewed by Bernardo Botella and Francisco Guerrero for CASSANDRASC-134
4341f1e00f92b9da132e13acd4eb4760f1d89e3f | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-05-24 18:11:14-07:00
CASSANDRASC-133: Allow optional reason to abort restore jobs (#124)
Abort restore job request can include the reason for the operation now optionally. The reason is logged and persisted for the restore job.
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-133
7ecef7c3c2e2137d4cb045ccf429703cacf96370 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-05-15 21:17:06-07:00
CASSANDRASC-131: Fix SidecarLoadBalancingPolicy unexpectedly removing local node and improve CI stability (#122)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-131
385ab5444884db09e7e018e24416a13198ea5139 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-05-10 13:23:44-07:00
CASSANDRASC-130: Improve integration test speed (#121)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-130
f4014c06d7668541010d59cc932970e9ebfc36f5 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-05-10 13:17:04-07:00
CASSANDRA-19626 Fix NullPointerException when reading static column with null values (#58)
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19626
466e7bf5e160d1667c12ea1de1b79ba27670aba4 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-05-03 10:11:18-07:00
CASSANDRA-19616: Integrate with the latest sidecar client (#56)
The patch updates the analytics code to consume the latest sidecar client after CASSANDRASC-127
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRA-19616
55866caa2c4601b1d59a8532a97310a9e819931f | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-05-03 08:21:16-07:00
CASSANDRASC-127: Reduce implementations accessible from client (#118)
* CASSANDRASC-127: Reduce implementations accessible from client
The patch breaks the existing `common` into `client-common` and `server-common`, where `client-common` is an API dependency of `server-common`. Sidecar clients only have access to `client-common`.
It also modernizes the WorkerExecutor api usage.
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-127
22b6df435936bdf0a616cf811c9a84821d4eae3c | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-04-25 12:40:44-07:00
CASSANDRASC-126: Make RestoreJobDiscoverer less verbose (#116)
Avoid logging the identical restore job info repeatedly
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-126
aea798dc7e517af520a403d4d86f3bc6bed65092 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-04-22 15:46:08-07:00
CASSANDRA-19563: Support bulk write via S3 (#53)
This commit adds a configuration (writer) option to pick a transport other than the previously-implemented "direct upload to all sidecars" (now known as the "Direct" transport). The second transport, now being implemented, is the "S3_COMPAT" transport, which allows the job to upload the generated SSTables to an S3-compatible storage system, and then inform the Cassandra Sidecar that those files are available for download & commit.
Additionally, a plug-in system was added to allow communications between custom transport hooks and the job, so the custom hook can provide updated credentials and out-of-band status updates on S3-related issues.
Co-Authored-By: Yifan Cai <ycai@apache.org>
Co-Authored-By: Doug Rohrer <drohrer@apple.com>
Co-Authored-By: Francisco Guerrero <frankgh@apache.org>
Co-Authored-By: Saranya Krishnakumar <saranya_k@apple.com>
Patch by Yifan Cai, Doug Rohrer, Francisco Guerrero, Saranya Krishnakumar; Reviewed by Francisco Guerrero for CASSANDRA-19563
686d9e8699bd0a56857e24523d883120023b9841 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-04-15 09:48:59-07:00
CASSANDRASC-119: Fix flaky test due to port conflicts (#113)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-119
9d61f3568b7961de3f95e567d461e5103441a161 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-04-13 07:04:52-07:00
CASSANDRASC-118: Support sending json request (#112)
Patch by Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-118
690101840d4d8f9c656bb0ca114f6619af80e1cf | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-04-08 14:33:50-07:00
CASSANDRA-19526: Optionally enable TLS in the server and client for Analytics testing
All integration tests today run without TLS, which is generally fine because they run locally. However,
it is helpful to be able to start up the sidecar with TLS enabled in the integration test framework so
that third-party tests could connect via secure connections for testing purposes.
Co-authored-by: Doug Rohrer <drohrer@apple.com>
Co-authored-by: Francisco Guerrero <frankgh@apache.org>
Patch by Doug Rohrer, Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19526
cbbf33d001b6f953be5654f00d7dfb54011a7619 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-04-03 11:08:32-07:00
CASSANDRA-19519: Migrate remaining integration tests to the single dtest cluster per class model (#49)
Additionally, we remove the usused test framework code after migrating the tests
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19519
295358095db80ced4b8f54f603f7bd9833a8f175 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-04-02 16:26:40-07:00
CASSANDRA-19513: Refactor Cassandra bridge (#48)
This commit splits the bridge implementation from the shaded `cassandra-all` library. This separation
allows for better integration of a different `cassandra-all` implementations. Additionally, it better
separates the actual bridge code from the Cassandra code.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19513
c00c454d698e5a29caf58e61ed52ab48d08fd7fe | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-04-01 12:11:52-07:00
CASSANDRA-19507 Fix bulk reads of multiple tables that potentially have the same data file name (#47)
When reading multiple data frames using bulk reader from different tables, it is possible to encounter a data
file name being retrieved from the same Sidecar instance. Because the `SSTable`s are cached in the `SSTableCache`,
it is possible that the `org.apache.cassandra.spark.reader.SSTableReader` uses the incorrect `SSTable` if it was
cached with the same `#hashCode`.
In this patch, the equality takes into account the keyspace, table, and snapshot name.
Additionally, we implement the `hashCode` and `equals` method in `org.apache.cassandra.clients.SidecarInstanceImpl` to utilize the `SSTableCache` correctly. Once the methods are implemented, the issue originally described in JIRA is surfaced.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19507
d28442ae712c1597052493aa3d2353a2de2495c2 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-03-27 13:32:39-07:00
CASSANDRA-19500 Fix XXHash32Digest calculated digest value (#46)
This PR bumps the Sidecar version to the current latest HEAD of Sidecar. Bumping the
version surfaced an issue with the way we are producing digest strings for the XXHash32
implementation. The hash value is not masked and this causes the negative sign to be
forwarded producing the incorrect hash result.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19500
164243e78f1557a34bc699ebc716b532781d6422 | Author: Arjun Ashok <arjun_ashok@apple.com>
| 2024-03-22 16:22:44-07:00
CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation (#41)
Patch by Arjun Ashok; Reviewed by Doug Rohrer, Yifan Cai, Francisco Guerrero for CASSANDRA-19418
ca1de4cc2e594266c1fca9f83cfcac38d9a21c45 | Author: Yifan Cai <ycai@apache.org>
| 2024-03-08 12:17:38-08:00
CASSANDRASC-114 Make hash algorithm implementation pluggable
patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-114
6ce33604bbd9acbee092ab3c4f7f11c0d434f730 | Author: Saranya Krishnakumar <saranya_k@apple.com>
| 2024-03-06 14:32:22-08:00
CASSANDRA-19424 Check for expired certificate during start up validation (#43)
patch by Saranya Krishnakumar; reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19424
d19a1b9a112614d378b42bb136cc84ae9cc6213a | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-03-06 13:57:00-08:00
CASSANDRASC-112 Fix ClosedChannelException when downloading from S3 (#103)
patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-112
a13532272051d4e4608f92d53bdd997103e8ea19 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-03-05 11:06:36-08:00
CASSANDRA-19452 Use constant reference time during bulk read process (#44)
patch by Yifan Cai; reviewed by Francisco Guerrero, James Berragan for CASSANDRA-19452
835b0f72b5e8a6fd5c9ac29b97a47942b97740b2 | Author: Yifan Cai <52585731+yifan-c@users.noreply.github.com>
| 2024-02-29 08:39:06-08:00
CASSANDRASC-110 Fix NPE thrown when getting StorageClient from pool (#101)
patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-110
46c35d0ef2efb66512133a7913df9936b0a80dc8 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-02-19 20:50:16-08:00
CASSANDRA-19411: Bulk reader fails to produce a row when regular column values are null
Bulk Reader won't emit a row when the regular column values are all `null`. For example,
a schema `PK` = `a`, `b` ; `CK` = `c`, `d` ; and columns = `e`, `f`.
| a | b | c | d | e | f |
| --- | --- | --- | --- | ---- | ---- |
| pk1 | pk2 | ck1 | ck2 | null | null |
When queried from Analytics bulk reader, it won't produce a row.
This issue also occurs when the projected regular column values are all `null`, where
other non-projected columns might have some values.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19411
c3e8803b3331bc7ef81797ac52a8417524f67edc | Author: Yifan Cai <ycai@apache.org>
| 2024-02-13 09:52:57-08:00
CASSANDRA-19285 Fix flaky Host replacement tests and shrink tests
The flakiness is caused by inspecting a class whose classloader is already closed. The fix is to include the those classes in the sharedClassLoader, so that the classLoader is not closed during the test.
patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRA-19285
2dcbca66a9f6a1e6b7cf4bd2c4f1368cc2ede86a | Author: Yifan Cai <ycai@apache.org>
| 2024-02-10 20:13:13-08:00
CASSANDRASC-103 Automated yaml type binding for deserialization
patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-103
fc08d45b283e701aa6d558e99cd18318394b0de7 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-01-31 14:35:34-08:00
CASSANDRA-19351 No longer need to synchronize on Schema.instance after Cassandra 4.0.12
We no longer need to synchronize on the `Schema.instance` in Analytics after the release of Cassandra
4.0.12, that includes a synchronization fix in https://issues.apache.org/jira/browse/CASSANDRA-18317.
This commit cleans up TODOs pending on that code being released.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19351
dc0e79b9c483562ec0920d69e886715eb329c426 | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-01-31 13:44:23-08:00
CASSANDRA-19369 Use XXHash32 for digest calculation of SSTables
This commit adds the ability to use the newly supported in Cassandra Sidecar XXhash32 digest algorithm.
The commit allows for backwards compatibility to perform MD5 checksumming, but it now defaults to XXHash32.
A new Writer option is added:
```
.option(WriterOptions.DIGEST.name(), "XXHASH32") // or
.option(WriterOptions.DIGEST.name(), "MD5")
```
This option defaults to XXHash32, when not provided, but it can be configured to use the legacy MD5 algorithm.
Path by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19369
ee454741363f3f693726af242c5ec37ad1480d60 | Author: Yifan Cai <ycai@apache.org>
| 2024-01-29 16:09:25-08:00
CASSANDRASC-99 Break restore job into stage and import phases and persist restore slice status on phase completion
patch by Yifan Cai; reviewed by Doug Rohrer, Francisco Guerrero for CASSANDRASC-99
e0ae9d7484e242f6af495aac2cb4d8dc121fba89 | Author: Yifan Cai <ycai@apache.org>
| 2024-01-24 15:38:41-08:00
CASSANDRA-19334 Upgrade to Cassandra 4.0.12 and remove BufferMode and BatchSize options
In cassandra-all:4.0.12, improvements were made for the CQLSSTableWriter. The sorted writer now can produce size-capped SSTables. It replaces the need for the unsorted sstable writer, which has to buffer and sort data on flushing. The dataset to write in the spark application is already sorted. By avoiding using the unsorted writer, it prevents wasting CPU time on sorting the sorted data. Since the sorted sstable writer does not need to buffer data, its size estimation is more accurate than the unsorted one, meaning the produced sstables files are closer to the expectation.
By removing the unsorted sstable writer, it no longer requires the RowBufferMode option.
By supporting size-capping in sorted writer, it no longer requires the BatchSize option.
Patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRA-19334
d949d8c2b9813c3e8429ece34c364a356bd7d6eb | Author: Francisco Guerrero <frankgh@apache.org>
| 2024-01-22 09:00:52-08:00
CASSANDRA-19275 Fix flaxy host replacement tests and shrink tests
This patch fixes flaky tests when a `BindException` occurs during cluster provisioning.
When a `BindException` is encountered, cluster provisioning is retried for up-to
`MAX_CLUSTER_PROVISION_RETRIES`.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19275
1b6052a8c3b04fca48f82e887c12bcfb2bae0bdb | Author: Yifan Cai <ycai@apache.org>
| 2024-01-19 10:52:50-08:00
CASSANDRASC-95 Add sidecar client changes for restore from S3
Patch by Saranya Krishnakumar, Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-95
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
fa6df8e2c09ad3d27bfe8c0ce016c839094630f6 | Author: Arjun Ashok <arjun_ashok@apple.com>
| 2024-01-16 12:24:02-08:00
CASSANDRA-19272 Add new writer option for blocklisted instances and corresponding integration tests
Patch by Arjun Ashok; Reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19272
6deef0107ae4cf1c18ca67bfbfd225619bf54ce1 | Author: Yifan Cai <ycai@apache.org>
| 2024-01-11 13:47:47-08:00
CASSANDRASC-93 Define routing order for http routes
Patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-93
550bdfa1c6082537e2cfb93449128a61dbe3a1fb | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-12-19 12:50:43-08:00
CASSANDRA-19251 Speed up integration tests
This commit introduces an opinionated way to run integration tests where a test class
reuses the same in-jvm dtest cluster, and it offers certain ordering that help running
tests faster.
The test setup does the following:
- Find the Cassandra version to run
- Provision a cluster for the test
- Initialize schemas required for tests
- Start the Sidecar service
The above approach guarantess that Sidecar is ready once the setup method completes,
which means we no longer need to spend time waiting for schema propagation. This
optimization also helps in reducing test time.
The drawback of this approach is that if we need the cluster to be in some state for
testing, for example a node needs to be in joining state while executing the bulk test
then, that cluster can only be used for tests in that state. Which means that testing
different states of the cluster requires a new test class.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19251
d61e44f78fa4ba5ec395e1e39c507d666fddefd1 | Author: Yuriy Semchyshyn <yuriy@semchyshyn.com>
| 2023-11-29 17:49:29-06:00
CASSANDRA-19377 Startup Validation Failures when Checking Sidecar Connectivity
patch by Yuriy Semchyshyn; reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19377
acea8725c0ff62cfcf538515f4b94a97e23414b4 | Author: Saranya Krishnakumar <saranya_k@apple.com>
| 2023-11-27 09:12:26-08:00
CASSANDRASC-92 Add restore from S3 feature to Sidecar
Patch by Saranya Krishnakumar, Yifan Cai; Reviewed by Francisco Guerrero for CASSANDRASC-92
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
c73c76498b0c2b36705025de6b0b2a7bb38e758b | Author: Doug Rohrer <drohrer@apple.com>
| 2023-11-20 10:54:46-05:00
CASSANDRA-19048 - Audit table properties passed through Analytics CqlUtils
The following properties have an effect on the files generated by the
bulk writer, and therefore need to be retained when cleaning the table
schema:
bloom_filter_fp_chance
cdc
compression
default_time_to_live
min_index_interval
max_index_interval
Additionally, this commit adds tests to make sure all available TTL
paths, including table default TTLs and constant/per-row options, work
as designed.
Patch by Doug Rohrer; Reviewed by Francisco Guerrero Hernandez, Yifan Cai,
Dinesh Joshi for CASSANDRA-19048
c7c3bbca2c7cb415b39689e924fa2357c239f043 | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-11-14 16:28:14-08:00
CASSANDRA-19031: Fix bulk writing when using identifiers that need quotes
Cassandra treats all identifiers as lower case unless explicitly quoted by the users,
(i.e. keyspace names, table names, column names, etc). We can define a case-sensitive
identifier or we can use a reserved word as an identifier by quoting it during DDL
creation.
In the analytics library, bulk writing fails when we encounter these identifiers. In
this commit, we fix the issue by property propagating the information about whether
identifiers need to be quoted by exposing a new dataframe option (`quote_identifiers`).
When set to `true`, it will _maybe_ quote the keyspace/table/column names and it will
properly be able to write data when using mixed-case or reserved words in the
identifiers.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19031
457b36bcb3c8a865cca83ca6c402246798113ab4 | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-11-13 16:16:36-08:00
CASSANDRA-19024 Fix bulk reading when using identifiers that need quotes
Cassandra treats all identifiers as lower case unless explicitly quoted by the users,
(i.e. keyspace names, table names, column names, etc). We can define a case-sensitive
identifier or we can use a reserved word as an identifier by quoting it during DDL
creation.
In the analytics library, bulk reads fail when we encounter these identifiers. In this,
commit, we fix the issue by properly propagating information about whether identifiers
need to be quoted by exposing a new data frame option (`quote_identifiers`). When set to
`true`, it will maybe quote the keyspace/table and it will properly be able to read data
when these situations are encountered.
Patch by Francisco Guerrero; Reviewed by Yifan Cai for CASSANDRA-19024
0aaf5659028dd874c8d666c636f11eae63c429e6 | Author: Arjun Ashok <arjun_ashok@apple.com>
| 2023-10-09 07:53:40-07:00
CASSANDRA-18852 - Changes to make bulk writer resilient to cluster resize operations
Patch by Arjun Ashok, Saranya Krishnakumar; Reviewed by Yifan Cai, Francisco Guerrero, Doug Rohrer for CASSANDRA-18852
Co-authored-by: Arjun Ashok <arjun_ashok@apple.com>
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
eb30005251cd8c10732ecab8365ebaa45f5fcbde | Author: Ekaterina Dimitrova <ekaterina.dimitrova@datastax.com>
| 2023-08-29 11:52:08-04:00
Upgrade caffeine cache and fix CIDR permissions cache invalidation
patch by Ekaterina Dimitrova; reviewed by Jacek Lewandowski, Ben Manes, Yifan Cai, Shailaja Koppu for CASSANDRA-18805
6ffa43f68b8d10ca84d4a00bf81269527b4e14df | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2023-08-25 11:10:48-06:00
Support Dynamic Port Allocation for in-jvm dtest framework
patch by Francisco Guerrero; reviewed by Dinesh Joshi, Jon Meredith, Yifan Cai for CASSANDRA-18722
f24951ab6ea2b1e9af4013b030675c70d31adb90 | Author: Yuriy Semchyshyn <yuriy@semchyshyn.com>
| 2023-08-14 14:09:12-05:00
CASSANDRA-18810: Cassandra Analytics Start-Up Validation
Patch by Yuriy Semchyshyn; Reviewed by Dinesh Joshi, Francisco Guerrero, Yifan Cai for CASSANDRA-18810
9c796dfb272daa3ce57a2dc5cbeadd9273e1ac72 | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2023-07-28 09:26:20-07:00
Skip ColumnFamilyStore#topPartitions initialization when client or tool mode
This commit skips the initialization of `topPartitions` in `org.apache.cassandra.db.ColumnFamilyStore`
when running in client or tool mode. The `TopPartitionTracker` class will attempt to query the system
keyspace, which when running in client or tool mode will not be part of the KeyspaceMetadata. This
causes a warning to be printed out with a stacktrace that can be misleading. The warning is similar to
this:
```
WARN org.apache.cassandra.db.SystemKeyspace: Could not load stored top SIZES partitions for ...
org.apache.cassandra.db.KeyspaceNotDefinedException: keyspace system does not exist
at org.apache.cassandra.schema.Schema.validateTable(Schema.java:xxx) ~[?:?]
at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
at org.apache.cassandra.cql3.statements.SelectStatement$RawStatement.prepare(SelectStatement.java:xxx) ~[?:?]
at org.apache.cassandra.cql3.QueryProcessor.parseAndPrepare(QueryProcessor.java:xxx) ~[?:?]
...
```
In this commit, we check whether we run in client or tool mode, and skip initialization
of `topPartitions` in those cases.
Patch by Francisco Guerrero; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18697
82b3c0a79c9322142738a4ec2ff7d4d4c0be2370 | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-07-25 12:41:10-07:00
CASSANDRA-18692 Fix bulk writes with Buffered RowBufferMode
When setting Buffered RowBufferMode as part of the `WriterOption`s,
`org.apache.cassandra.spark.bulkwriter.RecordWriter` ignores that configuration and instead
uses the batch size to determine when to finalize an SSTable and start writing a new SSTable,
if more rows are available.
In this commit, we fix `org.apache.cassandra.spark.bulkwriter.RecordWriter#checkBatchSize`
to take into account the configured `RowBufferMode`. And in specific to the case of the
`UNBUFFERED` RowBufferMode, we check then the batchSize of the SSTable during writes, and for
the case of `BUFFERED` that check will take no effect.
Co-authored-by: Doug Rohrer <doug@therohrers.org>
Patch by Francisco Guerrero, Doug Rohrer; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18692
014db08a79f00ef0d94e6855779e398c9dc689c1 | Author: James Berragan <jberragan@apple.com>
| 2023-07-19 12:23:07-07:00
CASSANDRA-18683: Add PartitionSizeTableProvider for reading the compressed and uncompressed sizes of all partitions in a table by utilizing the SSTable Index.db files
Patch by James Berragan; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18683
5597b7ec3c87d2dae202bb4f4ac583aeff902231 | Author: Yifan Cai <ycai@apache.org>
| 2023-07-12 21:55:15-07:00
CASSANDRASC-65 Enrich RetriesExhaustedException to have more information for better visibility
patch by Yifan Cai; reviewed by Dinesh Joshi, Francisco Guerrero for CASSANDRASC-65
109d9abf8d2c7816378ed078b6151baa50fc4ab8 | Author: Yifan Cai <ycai@apache.org>
| 2023-06-27 15:44:08-07:00
CASSANDRASC-62: Fix the uncaught exception in client when processing response
patch by Yifan Cai; reviewed by Dinesh Joshi, Francisco Guerrero for CASSANDRASC-62
69766bca399cc779e0f2f8e859e39f7e29a17b7a | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-06-27 10:03:56-07:00
CASSANDRA-18662: Fix cassandra-analytics-core-example
This commit fixes the `SampleCassandraJob` available under the `cassandra-analytics-core-example`
subproject.
Fix checkstyle issues
Fix serialization issue in SidecarDataTransferApi
The `sidecarClient` field in `SidecarDataTransferApi` is declared as transient,
this is causing NPEs coming from executors while trying to perform an SSTable
upload.
This commit completely avoids serializing the `dataTransferApi` field in the
`CassandraBulkWriterContext`, and lazily initializing it during the `transfer()`
method invocation. We guard the initialization to a single thread by making the
`tranfer()` method synchronized. The `SidecarDataTransferApi` can be recreated
when needed using the already serialized `clusterInfo`, `jobInfo`, and `conf`
fields.
Fix setting ROW_BUFFER_MODE to BUFFERED
patch by Francisco Guerrero; reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18662
9523a38b3f1b5bc4313e2949896ddc1fff58afbe | Author: jkonisa <jkonisa@apple.com>
| 2023-06-15 13:31:01-07:00
CASSANDRA-18605 Adding support for TTL & Timestamps for bulk writes
This commit introduces a new feature in Spark Bulk Writer to support writes with
constant/per_row based TTL & Timestamps.
Patch by Jyothsna Konisa; Reviewed by Dinesh Joshi, Francisco Guerrero, Yifan Cai for CASSANDRA-18605
deebdf97ad01f23550d7d3b42d98c7bf111e2f95 | Author: Doug Rohrer <drohrer@apple.com>
| 2023-06-14 13:33:29-04:00
CASSANDRA-18759: Use in-jvm dtest framework from Sidecar for testing
This commit introduces the use of the in-jvm dtest framework for testing
Analytics workloads. It can spin up a Cassandra cluster, including the
necessary Sidecar process, to test writing to and reading from Cassandra
using the analytics library.
Additional changes made in this commit include
* Use concurrent collections in MockBulkWriterContext (Fixes flaky test StreamSessionConsistencyTest)
The StreamSessionConsistency test uses the MockBulkWriter context, but it wasn't originally used
(before this test was added) in a multi-threaded environment. Because of this, it would occasionally
throw ConcurrentModificationExceptions, which would cause the stream test to fail in a
non-deterministic way. This commit adds the use of concurrent/synchronous collections to the
MockBulkWriterContext to make sure it doesn't throw these spurious errors.
* Make the StartupValidation system thread-safe by using TreadLocals
instead of static collections, and clearing them once validation is
complete.
Patch by Doug Rohrer; Reviewed by Dinesh Joshi, Francisco Guerrero, Yifan Cai for CASSANDRA-18759
933137fb14d4ebfe38d0d2c8e23b765aa1c89333 | Author: Shailaja Koppu <s_koppu@apple.com>
| 2023-06-13 17:18:29+01:00
CIDR filtering authorizer
Patch by Shailaja Koppu, Yifan Cai; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18592
Co-Authored-By: Yifan Cai <ycai@apache.org>
f0fae2deeee20df15ac1105af2163af2a7e7953d | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-06-08 12:40:22-07:00
CASSANDRA-18578 Add circleci configuration yaml for Cassandra Analytics
This commit adds the CircleCI configuration yaml to test against all the existing
profiles
- cassandra-analytics-core-spark2-2.11-jdk8
- cassandra-analytics-core-spark2-2.12-jdk8
- cassandra-analytics-core-spark3-2.12-jdk11
- cassandra-analytics-core-spark3-2.13-jdk11
Patch by Francisco Guerrero; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18578
ee1c83722bfb1155bef762cdfb2c86034857f2d0 | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-06-07 12:40:50-07:00
CASSANDRA-18574: Fix sample job documentation after Sidecar changes
This commit fixes the README file with documentation to setup and run the Sample job provided in the repository.
During Sidecar review, there was a suggestion to change the yaml property `uploads_staging_dir` to `staging_dir`.
That change however was not reflected as part of the sample job README.md.
patch by Francisco Guerrero; reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18574
7764214d1fb44fb6139a622f403bb05610e8f7b1 | Author: Francisco Guerrero <frankgh@apache.org>
| 2023-05-24 14:21:59-07:00
CASSANDRA-18548: Add the .asf.yaml file
This commit adds the .asf.yaml file to control notifications and github settings
for the Cassandra Analytics project.
Patch by Francisco Guerrero; Reviewed by Brandon Williams, Yifan Cai for CASSANDRA-18548
b87b0edd310d1ef93c507bbbb1ae51e1b0b319c6 | Author: Francisco Guerrero <francisco.guerrero@apple.com>
| 2023-05-23 13:56:48-07:00
CASSANDRA-18545: Provide a SecretsProvider interface to abstract the secret provisioning
This commit introduces the SecretsProvider interface that abstracts the secrets provisioning.
This way different implementations of the SecretsProvider can be used to provide SSL secrets
for the Analytics job. We provide an implementation, SslConficSecretsProvider, which provides
secrets based on the configuration for the job.
Patch by Francisco Guerrero; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-18545
f078c02cb58bddd735490b07548f7352f0eb09aa | Author: jkonisa <jkonisa@apple.com>
| 2023-05-22 22:15:22-07:00
Adding Mutual TLS authenticators for client & internode connections
Patch by Jyothsna Konisa & Dinesh Joshi; reviewed by Yifan Cai, Jon Meredith,
Yuki Morishita & Dinesh Joshi for CASSANDRA-18554
Co-Authored-By: Dinesh Joshi <djoshi@apache.org>
38cdacb2e7418e2aefbcffb1754dcd324c46028d | Author: Dinesh Joshi <djoshi@apache.org>
| 2023-05-19 15:34:05-07:00
CEP-28: Implement Bulk API endpoints and introduce the Sidecar Client to Support Cassandra Analytics
This commit implements the remaining endpoints needed to perform Bulk Analytics operations that allow
reading and writing data from Cassandra in Bulk. The new endpoints include:
- Endpoint to create snapshots
- Endpoint to clear a snapshot
- Endpoint to upload SSTable components
- Endpoint to clean up uploads for SSTable components
- Endpoint to import SSTable components
- Endpoint to retrieve gossip info
- Endpoint to retrieve the time skew for the server
- Endpoint to retrieve the ring information
Sidecar Client
Introduces the fully featured sidecar client to access Cassandra Sidecar endpoints.
It offers support for retries and Sidecar instance selection policies. The client
project itself is technology-agnostic, but we provide a vertx implementation for
the `HttpClient`. The Sidecar vertx-client can be published as a shaded-jar to be
consumed by clients where the dependencies can cause issues, especially in environments
where the dependencies are not always controlled by the consumers (for example Spark).
Patch by Doug, Francisco, Saranya, Yifan, Dinesh; reviewed by Dinesh Joshi and Yifan Cai for CASSANDRA-16222
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Francisco Guerrero <francisco.guerrero@apple.com>
Co-authored-by: Doug Rohrer <drohrer@apple.com>
Co-authored-by: Dinesh Joshi <djoshi@apache.org>
1633cd9c6c3d88d5c66825fab76a369266509f7e | Author: Dinesh Joshi <djoshi@apache.org>
| 2023-05-19 14:57:47-07:00
CEP-28: Apache Cassandra Analytics
This is the initial commit for the Apache Cassandra Analytics project
where we support reading and writing bulk data from Apache Cassandra from
Spark.
Patch by James Berragan, Doug Rohrer; Reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-16222
Co-authored-by: James Berragan <jberragan@apple.com>
Co-authored-by: Doug Rohrer <drohrer@apple.com>
Co-authored-by: Saranya Krishnakumar <saranya_k@apple.com>
Co-authored-by: Francisco Guerrero <francisco.guerrero@apple.com>
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Jyothsna Konisa <jkonisa@apple.com>
Co-authored-by: Yuriy Semchyshyn <ysemchyshyn@apple.com>
Co-authored-by: Dinesh Joshi <djoshi@apache.org>
26c374da4f03e4a6b64e414805cd92f3eb0a36c6 | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2023-03-09 12:11:20-08:00
Synchronize CQLSSTableWriter#build on the Schema.instance object
In this commit the `org.apache.cassandra.io.sstable.CQLSSTableWriter#build` method synchronizes on the
`Schema.instance` object (instead of the `CQLSSTableWriter.class`) to prevent concurrent schema operations
to fail when the offline tools also updates the schema.
For example, a table creation operation, which modifies the keyspace tables metadata, might end up
missing the update when a concurrent call to the `CQLSSTableWriter#build` method is accessing the
singleton Schema instance.
Patch by Francisco Guerrero, reviewed by Yifan Cai, Maxwell Guo, Alex Petrov for CASSANDRA-18317.
6224c618496e06390b28276812844aabe0182e64 | Author: Yifan Cai <yifan_cai@apple.com>
| 2022-12-01 14:44:56-08:00
CASSANDRASC-49: Set the copyright profile on generating idea project
Automatically add the ApacheV2 license header to new files
patch by Yifan Cai; reviewed by Dinesh Joshi for CASSANDRASC-49
ccada788c47882bfb10d7cb86f7f39c9865428f2 | Author: Stefan Miklosovic <smiklosovic@apache.org>
| 2022-11-30 21:31:43+01:00
Print exception message without stacktrace when nodetool commands fail on probe.getOwnershipWithPort()
Consequently, there is also alignement of nodetool ring command returning
exit code 1 in case there is unrecoverable exception thrown,
same as was already done for status and describecluster commands.
patch by Stefan Miklosovic; reviewed by Brandon Williams and Yifan Cai for CASSANDRA-18079
145dbd1f0875ae4c54392125e09ed05153c0dd8f | Author: Saranya Krishnakumar <saranya_k@apple.com>
| 2022-10-19 16:58:41-07:00
Add flag to exclude nodes from local DC when running nodetool rebuild
Patch by Saranya Krishnakumar; reviewed by Dinesh Joshi, Marcus Eriksson, Yifan Cai for CASSANDRA-17870
75194201f1f06d120f246f6fad025ca5f672943d | Author: Bernardo Botella Corbi <contacto@bernardobotella.com>
| 2022-10-10 09:08:16-07:00
Fix quoting in toCqlString methods of UDTs and aggregates
patch by Bernardo Botella Corbi, reviewed by Stefan Miklosovic, Benjamin Lerer and Yifan Cai for CASSANDRA-17918
d0b9532f2b87a17a0508d0637556f2f3e8d0fd94 | Author: Josh McKenzie <jmckenzie@apache.org>
| 2022-08-22 15:04:19-04:00
Add the ability for operators to loosen the definition of "empty" for edge cases
Patch by David Capwell; reviewed by Josh McKenzie, Yifan Cai, and Sam Tunnicliffe for CASSANDRA-17842
Co-authored-by: David Capwell <dcapwell@apache.org>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
83c169ec9e36324f27bf562951362f4a03c3c688 | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2022-08-19 10:20:57-07:00
Fix BulkLoader to load entireSSTableThrottle and entireSSTableInterDcThrottle
patch by Francisco Guerrero; reviewed by Ekaterina Dimitrova, Yifan Cai for CASSANDRA-17677
4526b3fcbde22d09065820286dd434d93ecc89ba | Author: Josh McKenzie <jmckenzie@apache.org>
| 2022-08-16 14:19:46-04:00
Add ability to log load profiles at fixed intervals
Patch by Yifan Cai; reviewed by Josh McKenzie, Dinesh Joshi, and Chris Lohfink for CASSANDRA-17821
Co-authored-by: Yifan Cai <ycai@apache.org>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
09b282d1fdd7d6d62542137003011d144c0227be | Author: Josh McKenzie <jmckenzie@apache.org>
| 2022-08-11 14:02:27-04:00
Rate-limit new client connection auth setup to avoid overwhelming bcrypt
Patch by Chris Lohfink; reviewed by Caleb Rackliffe, Yifan Cai, and Josh McKenzie for CASSANDRA-17812
Co-authored-by: Chris Lohfink <clohfink@apple.com>
Co-authored-by: Josh McKenzie <jmckenzie@apache.org>
9a4a67782311e4a6ffe3aad0516c33016f187c54 | Author: Bernardo Botella Corbi <contacto@bernardobotella.com>
| 2022-08-01 10:57:04-07:00
Removed Python < 2.7 support from formatting.py
Patch by Bernardo Botella Corbi, reviewed by Brad Schoening, ycai and
brandonwilliams for CASSANDRA-17694
7119cf8be1c32fa6ac015456965ae27a17cd828c | Author: Jyothsna Konisa <jkonisa@apple.com>
| 2022-06-30 09:52:50-07:00
Fix TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
patch by Jyothsna Konisa; reviewed by Jon Meredith, Yifan Cai for CASSANDRA-17700
a9725b681b948f2122f3d48b96a5c4e7403d2c39 | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2022-06-29 11:15:10-07:00
Fix AbstractCell#toString throws MarshalException for cell in collection
patch by Francisco Guerrero; reviewed by Caleb Rackliffe, Yifan Cai for CASSANDRA-17695
c8c8635a4c902ef051a46845919a5430f8d71e3f | Author: Andrés de la Peña <a.penya.garcia@gmail.com>
| 2022-06-17 11:45:43+01:00
Add ability to read the TTLs and write times of the elements of a collection and/or UDT
patch by Andrés de la Peña; reviewed by Yifan Cai and Benjamin Lerer for CASSANDRA-8877
ed3901823a5fe9f8838d8b592a1b7703b12e810b | Author: Jyothsna Konisa <jkonisa@apple.com>
| 2022-05-24 10:21:16-07:00
Adding support for TLS client authentication for internode communication
patch by Jyothsna Konisa; reviewed by Bernardo Botella, Francisco Guerrero, Jon Meredith, Maulin Vasavada, Yifan Cai for CASSANDRA-17513
ffc4c89c3df7ad0ae73ebefdcb7e15a2790c0a52 | Author: Doug Rohrer <drohrer@apple.com>
| 2022-05-17 15:09:16-04:00
Fix issue where frozen maps may not be serialized in the correct order
patch by Doug Rohrer, Francisco Guerrero and Yifan Cai; reviewed by Andrés de la Peña and Caleb Rackliffe for CASSANDRA-17623
Co-authored-by: Doug Rohrer <drohrer@apple.com>
Co-authored-by: Francisco Guerrero <frank.guerrero@gmail.com>
Co-authored-by: Yifan Cai <ycai@apache.org>
013acc641c5d487b07be5c082af1e85d26bd127f | Author: Bernardo Botella Corbi <contacto@bernardobotella.com>
| 2022-04-18 16:09:21-07:00
Rename DisableFlag class to EnableFlag on guardrails
patch by Bernardo Botella Corbi; reviewed by Andrés de la Peña, Josh McKenzie and Yifan Cai for CASSANDRA-17544
2fe1c304835ef39093b70cbb89107383be3c3ee9 | Author: Bernardo Botella Corbi <b_botellacorbi@apple.com>
| 2022-04-12 11:06:41-07:00
Fix flaky test LocalReadSizeWarningTest#failThresholdSinglePartition
patch by Bernardo Botella Corbi; reviewed by Andres de la Peña, Yifan Cai for CASSANDRA-17217
61be4d836213f708d9a29e59b9ef1df0bebef29a | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2022-03-16 01:31:00+01:00
expose gossip information in system_views.gossip_info virtual table
patch by Francisco Guerrero; reviewed by Stefan Miklosovic and Yifan Cai for CASSANDRA-17002
This commit adds a new virtual table that exposes the gossip information in tabular format.
The information is the same as the information presented through the `nodetool gossipinfo`
command, but the virtual table splits the version and value from `VersionedValue` into two
different columns. This is intented to help clients reading the vtable without the need of
parsing the version:value information (as it currently stands in gossipinfo).
The token value does not have a column. This is consistent with the gossipinfo output which
always renders ":<hidden>" for the Token value. Only the token_version column is available.
543608ba39d5803b963d14821abe193ff0796b4f | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2022-02-07 11:25:45-08:00
Instance failed to start up due to NPE in StartupClusterConnectivityChecker
patch by Francisco Guerrero; reviewed by Stefan Miklosovic, Yifan Cai for CASSANDRA-17347
945a4fc23ac1f60b8380be3b60aef89caf3daba2 | Author: Shailaja Koppu <s_koppu@apple.com>
| 2022-02-01 09:53:49-08:00
Add a virtual table for exposing prepared statements metrics
patch by Shailaja Koppu; reviewed by Ekaterina Dimitrova, Francisco Guerrero, Yifan Cai for CASSANDRA-17224
0dc5a289e8dd586150253d951e6e229480c0ffc8 | Author: Francisco Guerrero <frank.guerrero@gmail.com>
| 2022-01-14 16:13:00-08:00
Preserve tests that use BigInt numbers
Patch by Francisco Guerrero; reviewed by brandonwilliams and ycai for
CASSANDRA-17133
a41cdd64b217c451b5576abe2f455eaa7ec1f322 | Author: Francisco Guerrero <francisco.guerrero@apple.com>
| 2021-12-14 10:13:59-08:00
Allow column_index_size_in_kb to be configurable through nodetool
patch by Francisco Guerrero; reviewed by Dinesh Joshi, Yifan Cai for CASSANDRA-17121
33fd2dc817f56116af0b4b7d2cd5365ce52948d9 | Author: Caleb Rackliffe <calebrackliffe@gmail.com>
| 2021-11-17 16:43:00-06:00
Log queries that fail on timeout or unavailable errors up to once per minute by default
patch by Caleb Rackliffe and Marcus Eriksson; reviewed by David Capwell and Yifan Cai for CASSANDRA-17159
Co-authored-by: Caleb Rackliffe <calebrackliffe@gmail.com>
Co-authored-by: Marcus Eriksson <marcuse@apache.org>
77dde2a3c4b40da3d820d4852c572338acbf6dc9 | Author: Francisco Guerrero <francisco.guerrero@apple.com>
| 2021-11-09 13:32:15-08:00
Introduce separate rate limiting settings for entire SSTable streaming
patch by Francisco Guerrero; reviewed by Dinesh Joshi, Marcus Eriksson, Yifan Cai for CASSANDRA-17065
b29e1037e4da75dfd2a30ad10f8008b24941e37f | Author: David Capwell <dcapwell@apache.org>
| 2021-11-08 16:03:06-08:00
JVMStabilityInspector.forceHeapSpaceOomMaybe should handle all non-heap OOMs rather than only supporting direct only
patch by David Capwell; reviewed by Caleb Rackliffe, Yifan Cai for CASSANDRA-17128
1ad8bf67a9c82cbb5ff38e5cf785f9fe2516d009 | Author: Benjamin Lerer <b.lerer@gmail.com>
| 2021-11-04 16:47:16+01:00
Allow to aggregate by time intervals
Patch by Benjamin Lerer; review by Andres De la Pena and Yifan Cai for CASSANDRA-11871
The patch allow to use pure monotonic functions on the last attribute of the GROUP BY clause and introduce some floor functions that can be use to group by time range.
A function is pure if:
1. The function return values are identical for identical arguments
2. The function application has no side effects
A function is monotonic if it is either entirely nonincreasing or nondecreasing.
185893256f10c14207bffe49ae733fb1a970aec5 | Author: Francisco Guerrero <francisco.guerrero@apple.com>
| 2021-10-08 15:05:24-07:00
Allow GRANT/REVOKE multiple permissions in a single statement
patch by Francisco Guerrero; reviewed by Benjamin Lerer and Yifan Cai for CASSANDRA-17030
This commit allows GRANT/REVOKE statement to support multiple permissions with a single
statement. For example,
```
GRANT MODIFY, SELECT ON KEYSPACE field TO manager;
GRANT ALTER, DROP ON ROLE role1 TO role2;
```
29485e077526a1730e3ba0cb3ca268cfd060462a | Author: Yifan Cai <yifan_cai@apple.com>
| 2021-09-28 14:48:32-07:00
Fix broken sidecar gradle configuration due to Cassandra 40 version
Patch by Yifan Cai; reviewed by Dinesh Joshi and Saranya Krishnakumar for CASSANDRASC-31
fd69375af0d31dccf4e14404ac58c9ced0f64dd9 | Author: Yifan Cai <yifan_cai@apple.com>
| 2021-07-15 17:17:40-07:00
Fix CircleCI config to also build dtest jar from cassandra-4.0
patch by Yifan Cai; reviewed by Brandon Williams, Michael Semb Wever, Ekaterina Dimitrova for CASSANDRA-16805
2fba5c80ce7bf71d04c62043ffa1088b9e832d83 | Author: Marcus Eriksson, Jon Meredith <marcuse@apache.org>
| 2021-06-25 22:00:25-07:00
Avoid memoizing the wrong min cluster version during upgrades
patch by Marcus Eriksson, Jon Meredith; reviewed by Brandon Williams, Yifan Cai for CASSANDRA-16759
5f23e6d766a18782db82d955bf380239990d2c84 | Author: Caleb Rackliffe <calebrackliffe@gmail.com>
| 2021-06-23 14:25:12-05:00
Add isolated flush timer to CommitLogMetrics and ensure writes correspond to single WaitingOnCommit data points
patch by Caleb Rackliffe; reviewed by Yifan Cai for CASSANDRA-16701
964051b04767689a1a3a84493c0b4afe77d6edfc | Author: David Capwell <dcapwell@gmail.com>
| 2021-06-09 16:01:18-07:00
CASSANDRA-16728 - test_tombstone_failure_threshold_message assert checking if array == string, should check for array truthiness
patch by David Capwell; reviewed by Yifan Cai for CASSANDRA-16728
803a23b2f1b24701c22e7fadaf98c2c9a9491d35 | Author: Alexey Zotov <azotcsit@gmail.com>
| 2021-05-30 22:41:28-07:00
Fix LeveledCompactionStrategy compacts last level throw an ArrayIndexOutOfBoundsException
patch by Alexey Zotov; reviewed by Marcus Eriksson, Yifan Cai for CASSANDRA-15669
c52f50f046fd1a8e1c7f17d9a1952e2bd92aeb4c | Author: Mick Semb Wever <mck@apache.org>
| 2021-04-15 16:57:36+02:00
Restore running each test class in its own separate jvm and cassandra directory
(Remove test parallelism from ant build.xml)
patch by Mick Semb Wever; reviewed by Yifan Cai, David Capwell for CASSANDRA-16595
3edacd632c2f2c360ca1d931c6b4f6fd7326511f | Author: Ekaterina Dimitrova <ekaterina.dimitrova@datastax.com>
| 2021-04-05 15:00:50-04:00
Scheduled (delayed) schema pull tasks should not run after MIGRATION stage shutdown during decommission
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe and Yifan Cai for CASSANDRA-16495
efa25fc8d10bbfcefe14fc6f2a623b6a8b73b5cd | Author: Alexandre Dutra <adutra@users.noreply.github.com>
| 2021-03-26 19:14:39+01:00
Do not reassign System.out and System.err when running nodetool
authored by Alexandre Dutra; reviewed by Yifan Cai and Ekaterina Dimitrova for CASSANDRA-16533
c591978f4d265e42d0132418005ba63a99278c75 | Author: Yifan Cai <ycai@apache.org>
| 2021-03-15 20:40:25-07:00
Better handle legacy gossip application states during (and after) upgrades
Only remove duplicated legacy application states when `!hasMajorVersion3Nodes()` but always avoid duplicate status notifications.
patch by Yifan Cai; reviewed by Mick Semb Wever for CASSANDRA-16525
865b67b21d326728936f0b80681129c73a2e374a | Author: jacek-lewandowski <jacek.lewandowski@datastax.com>
| 2021-03-01 12:24:28+00:00
Fix ColumnFilter behaviour to prevent digest mitmatches during upgrades
patch by Jacek Lewandowski; reviewed by Andrés de la Peña and Yifan Cai for CASSANDRA-16415
12b610246bc42dc6af33abfe0885b2f989fc2c73 | Author: Brandon Williams <brandonwilliams@apache.org>
| 2021-01-20 12:46:23-06:00
Don't manually remove endpoints in distributed tests
Patch by brandonwilliams, reviewed by ycai and edimitrova for
CASSANDRA-16229
876ac8c6111697c8105899889bf794438110bc74 | Author: David Capwell <dcapwell@apache.org>
| 2020-12-02 13:35:56-08:00
When a table attempts to clean up metrics, it was cleaning up all global table metrics
patch by David Capwell; reviewed by Jon Meredith, Jordan West, Yifan Cai for CASSANDRA-16095
c6ea8fabf9f726a58b76ff6e41d21b650941f880 | Author: yifan-c <yc25code@gmail.com>
| 2020-11-17 16:06:03-08:00
Add dedicated tcp user timeout for streaming connection
patch by Yifan Cai; reviewed by Berenguer Blasi, David Capwell, Adam
Holmberg and by Benjamin Lerer for CASSANDRA-16143
4d1d024136dd5f4b3775a0666b0c30a65641d57f | Author: David Capwell <dcapwell@apache.org>
| 2020-11-10 10:00:20-08:00
jvm dtest is strict on properties which causes upgrade tests to fail
patch by David Capwell; reviewed by Caleb Rackliffe, Ekaterina Dimitrova, Jordan West, Yifan Cai for CASSANDRA-16256
beee6b441c71895ca7b2833631933a6a55b516c2 | Author: Caleb Rackliffe <calebrackliffe@gmail.com>
| 2020-10-30 16:24:52-05:00
AbstractArrayClusteringPrefix now properly handles null clustering elements from compact (or previously compact) tables
patch by Caleb Rackliffe; reviewed by Yifan Cai and Jordan West for CASSANDRA-16241
ce270081bc0bc8ffa0a7e1e5c04f30b5c1875a84 | Author: Yifan Cai <yc25code@gmail.com>
| 2020-10-29 10:03:56-07:00
Invalid serialized size for responses caused by increasing message time by 1ms which caused extra bytes in size calculation
patch by Yifan Cai; reviewed by David Capwell, Jon Meredith for CASSANDRA-16103
db5832120f0fe9ad799ca0b5b16bbcf6f7404580 | Author: Jon Meredith <jmeredithco@apple.com>
| 2020-10-22 10:54:55-07:00
In-JVM dtest - modify schema with stopped nodes and use yaml fragments for config
patch by Jon Meredith; reviewed by Alex Petrov, David Capwell, Dinesh Joshi, Yifan Cai for CASSANDRA-16152
f15c6b8c06c9588bd96615fabfc36b11857cd4bb | Author: Yifan Cai <yc25code@gmail.com>
| 2020-10-19 11:09:11-07:00
Backport CASSANDRA-16057: Should update in-jvm dtest to expose stdout and stderr for nodetool
patch by Yifan Cai; reviewed by Alex Petrov, David Capwell for CASSANDRA-16057
7e78059b38f367d1c98bb11df6ba78633c1516f0 | Author: Caleb Rackliffe <calebrackliffe@gmail.com>
| 2020-10-16 12:40:03-05:00
Throw BufferOverflowException from DataOutputBuffer for better visibility
patch by Caleb Rackliffe; Reviewed by Yifan Cai, Jordan West for CASSANDRA-16214
8333d0b0890f6299fac1fd219f58aee0c62cbf9c | Author: Marcus Eriksson <marcuse@apache.org>
| 2020-10-14 12:30:21+02:00
Reduce amount of allocations during batch statement execution
Patch by marcuse; reviewed by Benjamin Lerer, Michael Semb Wever and Yifan Cai
for CASSANDRA-16201
521a6e2aa9f8a4bc95dd13e768ec6de33cf6fa15 | Author: David Capwell <dcapwell@apache.org>
| 2020-10-12 09:30:41-07:00
Fixed a NullPointerException when calling nodetool enablethrift
patch by David Capwell; reviewed by Ekaterina Dimitrova, Jordan West, Yifan Cai for CASSANDRA-16127
5be83b6a72695253c552535d2b826209f144cc63 | Author: Alex Petrov <oleksandr.petrov@gmail.com>
| 2020-10-01 17:00:12+02:00
Fix NPEs when 3.0 messages get re-serialized for filtering on 4.0 nodes in in-JVM dtests.
Patch by Alex Petrov; reviewed by Yifan Cai and David Capwell for CASSANDRA-16157
ba63fa3c951cb5c18d0fa4f9483577c6e18389c4 | Author: Adam Holmberg <adam.holmberg@datastax.com>
| 2020-08-19 15:32:09-05:00
Fix flaky test ConnectionTest.testMessagePurging
patch by Adam Holmberg; reviewed by Yifan Cai and Benjamin Lerer for
CASSANDRA-15958
The patch fix 2 problems a race condition in InboundSocket.close when it
is called multiple times and the flakyness in ConnectionTest.testMessagePurging.
f3198c45067f753444ffafbc83b6aa563cb392ac | Author: yifan-c <yc25code@gmail.com>
| 2020-07-16 13:12:11-07:00
Verify sstable components on startup
Patch by Yifan Cai, reviewed by David Capwell and brandonwilliams for
CASSANDRA-15945
da95e4b3fb36294b5117846b8fbb8cdc01e427d5 | Author: Yifan Cai <yifan_cai@apple.com>
| 2020-04-28 13:51:15-07:00
Add tunable initial size and growth factor to RangeTombstoneList
Added `initial_range_tombstone_list_allocation_size` and
`range_tombstone_list_growth_factor` in config, which can be altered via JMX.
Patch By Michael Kjellman and Yifan Cai; Reviewed by Dinesh Joshi for CASSANDRA-15763
Co-Authored-By: Yifan Cai <yifan_cai@apple.com>
Co-Authored-By: Michael Kjellman <kjellman@apple.com>
563592801aad00e002f0a162d5e2625a4b0e8723 | Author: Doug Rohrer <doug@therohrers.org>
| 2019-12-11 21:33:24-05:00
Pass correct seed node through to InstanceConfig so it can set it properly when starting clusters on non-0 subnets.
Patch by Doug Rohrer, reviewed by Yifan Cai and David Capwell for CASSANDRA-15447.
5459c121d5d688ec1571dd72550bd6faaeb087eb | Author: Doug Rohrer <drohrer@apple.com>
| 2019-11-18 11:39:45-05:00
Support NodeTool in dtest
patch by Doug Rohrer and Yifan Cai; reviewed by David Capwell
and Dinesh Joshi for CASSANDRA-15429
Co-Authored-By: Doug Rohrer <drohrer@apple.com>
Co-Authored-By: Yifan Cai <yifan_cai@apple.com>
d3b3dcbb353de97220a11f55391babf149410905 | Author: yifan-c <yc25code@gmail.com>
| 2019-10-27 21:01:31-07:00
Separate exceptions for CAS write timeout exceptions caused by contention and unkown result
Patch by Yifan Cai; reviewed by Alex Petrov and Dinesh Joshi for CASSANDRA-15350