The issue is not "enough of it". Asbestos related health problems can develop as a result of a single exposure to a fibre of the "wrong" type ie shape and size, such that the fibre gets trapped in the lungs. Of course, the more asbestos there is the greater the chance that there will be some fibres of the "wrong" type. Not all asbestos contains these harmful fibres but there is unlikely to be any guarantee for older asbestos used in building or lagging so ALL asbestos should be treated with care. If asbestos-containing material is undisturbed and contained eg over-painted, then the hazard is minimal.
5 Precepts of Buddhism seem appropriate. Refrain from taking life. Refrain from taking that which is not given. Refrain from misconduct. Refrain from lying. Refrain from intoxicants which lead to loss of mindfulness