Thinking about removing gas fireplace

I would like some opinions here, as there are so many factors ive read about. Our living room is fairly big (Compared to our last house), but has a large bay type window on the front facing wall next to the front door, then the right wall has the staircase, and coat closet, and then the left wall has a fireplace thats taking up ALOT of space. Ideally, I would love this wall to be open to put a large entertainment center, and my TV, while having a sectional couch along the wall parallel to it. My inspector told me that for this fireplace to be used, some of the bricks/mortar inside would need to be replaced, along with basically the whole gas setup that sits inside. The gas to the fireplace is off at the meter i'm assuming, as there is no gas coming in the box. The bricks and mantle of the fireplace stick out about 12-14" from the wall, and the whole setup is VERY outdated. (We are young, and prefer more of a modern style)