No. It will not look strange. Most of the focus is going to be on the fire, and any gap behind the mantel and the wall will be in shadow, and not an area that people will be focusing on. However, if it is a concern for you, you can notch out your baseboards so that the mantel will sit flush. This is purely optional and up to you. Either way works.
