Well taking something down or breaking something to steal it is griefing, so as long as you return the stuff you have broken and "stolen" you can come back on..... But I'm not really being unreasonable there is a clear difference between griefing and stealing something out of a chest for example..