They were lucky that Forrest wasn't around...

Madison Parish served as a springboard for the siege of Vicksburg as it was located directly across the river from Vicksburg. It especially felt the brunt of the early stages of preparation when the Union Army used the surrounding country as a supply staging area, months before the actual siege began. 

The confiscation and destruction of personal and public property was especially heavy since there were almost no Confederate Troops in the area. The toll increased when the Union Army marched through the heart of the parish to cross the river below Vicksburg in order to attack from the rear

About the middle of 1862 federal Troops first appeared in the area, and on Christmas Day of that year General W. T. Sherman arrived by boat at Milliken's Bend. He immediately dispatched a detachment of about two thousand troops, principally infantry and cavalry, into the countryside. They destroyed private and public property, including the railroad bridges over the Tensas River and Bayou Macon, and burned the depot at Delhi.

A few remained in town for several hours, became intoxicated, and committed various outrages. The troops penetrated into the country for about thirty miles, unopposed and without loss of life, stole a number of Negroes, mules and cattle, burning several cotton gins and several hundred bales of cotton. "Vignettes" of the Civil War
By Francis McRae Ward