Not only does the Captain Marvel post-credit scene connect to The Avengers: Endgame, but yesterday during Disney's investor meeting, new footage from the Infinity War sequel also featured Brie Larson's Carol Danvers meeting Thor and more.

The new Avengers: Endgame footage from the Disney investor meeting revealed the Avengers sitting in a room at their headquarters with Captain Marvel (Brie Larson) also with them, which probably follows the Captain Marvel post-credit scene where Carol Danvers shows up.
Nebula speaks and says she thinks Thanos would go to "the garden" after completing his plan to wipe out half off all life in the universe.

Captain Marvel then offers the Avengers need to go to the garden and use the Infinity Stones to undo everything.
The Avengers: Endgame footage then switches gears to show Captain America, Thor, Black Widow and War Machine in space with Captain Marvel, Rocket Raccoon and Nebula.
Rocket then cracks a couple of jokes asking who has never been in space before to raise your hand and to not puke in his ship.

A user on Twitter also offers more about The Avengers: Endgame footage including how the Avengers are tracking Thanos and why Captain Marvel hasn't shown up before:
"The Avengers Endgame scene they showed was at Avengers HQ, Captain Marvel saying let's go kill Thanos. Nebula says she knows where he is. Rocket says biggest energy surge ever seen on Earth when he used the glove, but it was seen again on another planet shortly after," Scott Ladewig tweeted. "So that's where Thanos is retiring to. Quick exchange between Cap America, Natasha, Bruce, Rhodey, and Cap Marvel. Bruce asks what guarantee they have this ends differently. Cap Marvel says they have her this time."
Ladewig continues, "Rhodey asks where she was because they could have used her before. Cap Marvel says a lot of other planets out there need help and they don't have Avengers like Earth has. Thor gets up from sitting quietly in back, walks up to Cap Marvel, and summons his axe... Which whizzes right past Cap Marvel (she doesn't flinch) into his hand and he says 'I like this one.' Cut to the Milano, everybody buckled in, Cap Marvel in the copilot seat."
Ladewig finishes with, "Rocket asks who hasn't been to space, Cap America, Rhodey, Natasha raise hands, and Rocket warns them not to puke in his ship and off they go. Cool refection of stars and space stuff on Cap America's eye as we zoom in on him and they warp out of there."

The Avengers: Endgame opens April 26.