With the price of these you could stick a few in there (there would be a weight penalty, they are not exactly zero grams though for their weight the amount of compute is impressive but you then would also need more power).

Depending on what kind of drone you want to build (powered fixed wing, glider, quad, hex or octocopter) that might be more or less of an issue. Given the power draw and weight of an octocopter I don't think it would be much of an issue there but on a quadcopter, especially a very small one that might be prohibitive. But I'd focus on getting it to work before optimizing for size and weight.