Yeah, lots of variables. It would depend on the amount of radioactive contents and how well dispersed they become.
Very basically: The implosion of weapons grade material-that is to smash it together-is what is needed to begin the 'chain' and the splitting of atoms which release energy and the dangerous radioactive 'pollution' (think of a cars tail-pipe-it's releasing pollution from the effects or combusting fossil fuel). Dirty bombs are not like this.
Dirty bombs are packed with radioactive material, so the high explosives scatter that material. It would not have the fall-out pollution of a nuclear contribution. So, their effects are limited; but still, highly dangerous. It depends on the amount of both HE and "dirty" material packed within it.