Depends on how well you get on with them. By the sound of it (no pun intentended!) it's maybe a little dodgy. At the end of the day people have a right to go about their business and are bound to make a little bit of noise in the process, this is all part of being a human being. Make sure you don't make noise during unsociable hours and try and get a routine going so that your neighbours know when you are going to make a little noise. It's even worth going round and explaining that music is your life and you need to make a little noise now and again.
It'll be quite hard for the environmental health people to prove your breaching regulations so i wouldn't worry too much - **** 'em!!!