I think the material design u mention is those material use to design the app/web include color, icon, motion etc.

U can use whatever colors u like on the design but make sure it look comfortable/harmony. But sometime u still need follow certain colors depends on what client u work for. Let's say Maybank is the client, then u probably need to use Maybank corporate yellow & black colors on the design.