The "Underworld" as it is nicknamed by the community is an area of nothing but darkness but a bit of sand and a small pond of 1 stud deep water located under the mainland's mountains. There are 2 known ways that are know as seen in the last images. The entrances found both are in the mainland mountains.
If you do intend of going to the "Underworld", it is very difficult to get out afterwards, so be prepared to stay in there for a long time without contact from anyone and bring with you a lot of renewable supplies like a bucket, pickaxe, shovel, and seeds for wood and food.
You're going to need to bring every fruit seed, 1 Wheat, 1 Cotton Boll, a Shovel, a Stone Well, a Cotton Gin, a Lumbermill, and a Mill if you want to thrive in it with every item in the game inside it.
As of now it is not known if the Underworld was in fact made in purpose by the developers as an easter egg or if it is a bug not meant to be there or found. The devs have not acknowledged it, but it has not been fixed or removed.