Changing 6amp circuit breaker to a 32 one.

Hi I am UK based and have lived in my house 2 years. I don't have any problems with the electrics however whenever I turn the circuit breaker at the fuse box to do some work it won't turn back on so I replace the circuit breaker each time. The other day this happened again (I didn't end up doing any work so I know it's not anything I have done) I have just gone to order a new circuit breaker and noticed the newly broken one and an existing broken circuit breaker are both 6amp and the rest in the fuse box are 32amp (except the shower and cooker which are 40). Do you think they could be breaking because they are to low and that I need to replace it with a 32amp? This is for the downstairs lights which is 3 lights. The upstairs one is 32amp and supplies 4 lights. I have no electrics experience so please ask if you need further info. Thanks