In econometrics and statistics, the generalized method of moments (GMM) is a generic method for estimating parameters in statistical models.
Usually it is applied in the context of semiparametric models, where the parameter of interest is finite-dimensional, whereas the full shape of the distribution function of the data may not be known, and therefore maximum likelihood estimation is not applicable.
Try continuously updating weight matrix if possible (sometimes this method can take awhile).
If you can't use continuously updating, also try the iterative procedure, which is just like doing the two-step over and over again until your weight matrix converges over the updates.
My advice is to use the analytic gradient if you have it.