Leviticus 23:

26 Yahweh said to Moses,
27 "The tenth day of this seventh month is the Day of Atonement.”

Leviticus 23:32 (NIV): "It is a sabbath of rest for you, and you must deny yourselves.  From the evening of the ninth day of the month until the following evening you are to observe your sabbath."

So the beginning of the 10th day is scripturally described as from the evening of the 9th day.  If evening were at the beginning of a day, Leviticus 23:32 would force us to include all of the 9th day as part of the 10th day; therefore, evening must be a period at the end of a day.  Hence, Scripture shows us that the evening of a day is always a portion of time before the end of the day.

This seems to force a definition of “evening” as a period of the day before sunset, since the next day begins just after sunset.  This also seems to force a definition of the beginning part of a day just after sunset as “morning” (even though it will be getting dark and will continue to be dark until the sun rises at dawn).

Exodus 12:6 says to kill the Passover lamb on the 14th day of the 1st month in the evening.  Evening is a period of the day before sunset.  See above.

(Ref. 1): "In the evening" in the Hebrew would be said: "bain hahaarbayim" (literally translated "between the evenings”).  This phrase suggests the time, a point between the sun's beginning to decline toward the west and its setting at the horizon (approximately 3:00 p.m.).  Within the temple, the day was divided into quarters.  The quarter between 12:00 noon and 3:00 p.m. was called the minor evening oblation, while that between 3:00 p.m. and 6:00 p.m. was called the major evening oblation.  Therefore, "between the evenings" means between those two periods or 3:00 p.m.  The hours were counted from daylight (approximately 6:00 a.m.) until sundown (approximately 6:00 p.m.).  The ninth hour, when Yahshua breathed his last breath on the cross, would therefore be 3:00 p.m., the same time the Passover lamb had been slain in Egypt.

(1) "Rosh HaShanah and the Messianic Kingdom to Come" Hatikva Ministries 1989, Joseph Good.