[antlr-interest] plugin, ITokenScanner, nextToken --> SWT BUG
Nev
antlr at zavalon.com
Wed Apr 9 23:12:46 PDT 2008
Are you running on Windows. Likely to have \r\n at end of each line.
Nev
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of elekis
> Sent: Thursday, 10 April 2008 4:07 PM
> To: Jan van der Ven; ANTLRDev Interest
> Subject: Re: [antlr-interest] plugin, ITokenScanner, nextToken --> SWT BUG
>
> re-hi,
>
> I thing I found where is the prob, but I have no idea how to patch it.
> there is the code I wanna scan:
>
> FROM fslkdfjsl IMPORT lskfjslk;
> FROM fslkdfjl IMPORT ldsfkjdjkfl;
>
> easy isnt -it
>
> and there is what getNextToken, getoffset and getlength return.
> TOKEN -->FROM<--
> OFFSET: 0
> SIZE: 4
> --------------------------------
> TOKEN --> <--
> OFFSET: 4
> SIZE: 1
> --------------------------------
> TOKEN -->fslkdfjsl<--
> OFFSET: 5
> SIZE: 9
> --------------------------------
> TOKEN --> <--
> OFFSET: 14
> SIZE: 1
> --------------------------------
> TOKEN -->IMPORT<--
> OFFSET: 15
> SIZE: 6
> --------------------------------
> TOKEN --> <--
> OFFSET: 21
> SIZE: 1
> --------------------------------
> TOKEN -->lskfjslk<--
> OFFSET: 22
> SIZE: 8
> --------------------------------
> TOKEN -->;<--
> OFFSET: 30
> SIZE: 1
> --------------------------------
> PROBLEME HERE WHY SIZE = 2
> TOKEN -->
> <--
> SIZE: 2
>
> etc....etc...
>
> there is my switch and WS is define like that
> WS :(' '|'\r'|'\t'|'\u000C'|'\n')+;
>
> switch (antCurrentToken.getType()) {
> case LyaflplugLexer.WS:
> return Token.WHITESPACE;
> case LyaflplugLexer.TAG_POINT_VIRGULE:
> return Token.UNDEFINED;
> case LyaflplugLexer.TAG_END:
> case LyaflplugLexer.TAG_DEFINITION_MODULE:
> case LyaflplugLexer.TAG_FROM:
> case LyaflplugLexer.TAG_IMPORT:
> return keyword;
> case LyaflplugLexer.Ident:
> return ident;
> default:
> return Token.UNDEFINED;
> }
>
> What I don't understand is why the size is 2??? there only one \n. any
> idea??
> thanks
>
> a+++;
>
>
>
> Jan van der Ven a ?crit :
> > Dear Julien,
> >
> > I remember having the same kind of problems. Please check the way we
> > fixed it here:
> > http://quantum.cvs.sourceforge.net/quantum/quantum-
> plugin/src/com/quantum/editors/SQL2ScannerWithLexer.java?view=log
> >
> > If I remember correctly, the setRange is sometimes called with illegal
> > values for offset and/or length.
> >
> > Happy coding,
> >
> > Jan
> >
> > julien wrote:
> >> hi all, I try to make a plugin who can colorize some line like
> >>
> >> FROM lkfsjdl IMPORT lskdfjl;
> >> FROM lkfsjdl IMPORT lskdfjl;
> >>
> >> Based on my own grammar created with ANTLR 3.0.
> >>
> >> for that, I overrided ITokenScanner.
> >>
> >> public IToken nextToken() {
> >>
> >> antCurrentToken = lexer.nextToken();
> >> if (antCurrentToken.getType() == org.antlr.runtime.Token.EOF)
> >> return Token.EOF;
> >> switch (antCurrentToken.getType()) {
> >> case LyaflplugLexer.TAG_END:
> >> case LyaflplugLexer.TAG_DEFINITION_MODULE:
> >> case LyaflplugLexer.TAG_FROM:
> >> case LyaflplugLexer.TAG_IMPORT:
> >> return keyword;
> >> default:
> >> return ident;
> >> }
> >> }
> >>
> >> The problem is : if the file is empty then I can create twho line
> >>
> >> FROM lkfsjdl IMPORT lskdfjl;
> >> FROM lkfsjdl IMPORT lskdfjl;
> >>
> >> he color eatch word etc... everything is perfect but if I save and I
> >> reopen the file he bug cause theres are two line who are the same.
> >> (if there is only one, no prob).
> >>
> >> I ve tried to debug and he call every token. but after that there is
> >> a exception error Argument No valid.
> >>
> >> is any body has a idea: I there is the class code
> >> http://paste.ubuntu-nl.org/62601/
> >>
> >> thanks
> >>
> >> a++
> >>
> >> PS: this message is sended on antlr-interest and eclipse.platform
> >> -----------------------------------------------------------------------
> >>
> >> PPS: there is the stack.
> >>
> >>
> >> java.lang.IllegalArgumentException: Argument not valid
> >> at org.eclipse.swt.SWT.error(SWT.java:3547)
> >> at org.eclipse.swt.SWT.error(SWT.java:3481)
> >> at org.eclipse.swt.SWT.error(SWT.java:3452)
> >> at
> >> org.eclipse.swt.custom.StyledText.setStyleRanges(StyledText.java:7776)
> >> at
> >> org.eclipse.swt.custom.StyledText.setStyleRanges(StyledText.java:7870)
> >> at
> >>
> org.eclipse.jface.text.TextViewer.applyTextPresentation(TextViewer.java:44
> 29)
> >>
> >> at
> >>
> org.eclipse.jface.text.TextViewer.changeTextPresentation(TextViewer.java:4
> 481)
> >>
> >> at
> >>
> org.eclipse.jface.text.presentation.PresentationReconciler.applyTextRegion
> Collection(PresentationReconciler.java:580)
> >>
> >> at
> >>
> org.eclipse.jface.text.presentation.PresentationReconciler.processDamage(P
> resentationReconciler.java:569)
> >>
> >> at
> >>
> org.eclipse.jface.text.presentation.PresentationReconciler.access$3(Presen
> tationReconciler.java:565)
> >>
> >> at
> >>
> org.eclipse.jface.text.presentation.PresentationReconciler$InternalListene
> r.inputDocumentChanged(PresentationReconciler.java:119)
> >>
> >> at
> >>
> org.eclipse.jface.text.TextViewer.fireInputDocumentChanged(TextViewer.java
> :2685)
> >>
> >> at
> >> org.eclipse.jface.text.TextViewer.setDocument(TextViewer.java:2734)
> >> at
> >>
> org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:5
> 18)
> >>
> >> at
> >>
> org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:4
> 73)
> >>
> >> at
> >>
> org.eclipse.ui.texteditor.AbstractTextEditor.initializeSourceViewer(Abstra
> ctTextEditor.java:3786)
> >>
> >> at
> >>
> org.eclipse.ui.texteditor.AbstractTextEditor.createPartControl(AbstractTex
> tEditor.java:3332)
> >>
> >> at
> >>
> org.eclipse.ui.texteditor.StatusTextEditor.createPartControl(StatusTextEdi
> tor.java:53)
> >>
> >> at
> >>
> org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.createPartControl(Ab
> stractDecoratedTextEditor.java:378)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.j
> ava:661)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:42
> 6)
> >>
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartRefere
> nce.java:592)
> >>
> >> at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:299)
> >> at
> >>
> org.eclipse.ui.internal.presentations.PresentablePart.setVisible(Presentab
> lePart.java:179)
> >>
> >> at
> >>
> org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(Pr
> esentablePartFolder.java:268)
> >>
> >> at
> >>
> org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(Left
> ToRightTabOrder.java:65)
> >>
> >> at
> >>
> org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectP
> art(TabbedStackPresentation.java:400)
> >>
> >> at
> >>
> org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.j
> ava:1256)
> >>
> >> at
> >> org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1209)
> >> at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1604)
> >> at org.eclipse.ui.internal.PartStack.add(PartStack.java:499)
> >> at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:103)
> >> at org.eclipse.ui.internal.PartStack.add(PartStack.java:485)
> >> at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:112)
> >> at
> >>
> org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.
> java:63)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java
> :217)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:2
> 07)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:7
> 74)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManag
> er.java:673)
> >>
> >> at
> >>
> org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:634)
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.
> java:2737)
> >>
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:26
> 51)
> >>
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.access$13(WorkbenchPage.java:2643)
> >> at
> >> org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2595)
> >> at
> >> org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2590)
> >>
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2574)
> >>
> >> at
> >>
> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2565)
> >>
> >> at org.eclipse.ui.ide.IDE.openEditor(IDE.java:644)
> >> at org.eclipse.ui.ide.IDE.openEditor(IDE.java:603)
> >> at
> >>
> org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUt
> ility.java:286)
> >>
> >> at
> >>
> org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUt
> ility.java:139)
> >>
> >> at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:194)
> >> at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:175)
> >> at
> >>
> org.eclipse.jdt.ui.actions.SelectionDispatchAction.dispatchRun(SelectionDi
> spatchAction.java:268)
> >>
> >> at
> >>
> org.eclipse.jdt.ui.actions.SelectionDispatchAction.run(SelectionDispatchAc
> tion.java:244)
> >>
> >> at
> >>
> org.eclipse.jdt.internal.ui.packageview.PackageExplorerActionGroup.handleO
> pen(PackageExplorerActionGroup.java:316)
> >>
> >> at
> >>
> org.eclipse.jdt.internal.ui.packageview.PackageExplorerPart$5.open(Package
> ExplorerPart.java:613)
> >>
> >> at
> >>
> org.eclipse.jface.viewers.StructuredViewer$2.run(StructuredViewer.java:820
> )
> >>
> >> at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
> >> at org.eclipse.core.runtime.Platform.run(Platform.java:857)
> >> at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:46)
> >> at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:199)
> >> at
> >>
> org.eclipse.jface.viewers.StructuredViewer.fireOpen(StructuredViewer.java:
> 818)
> >>
> >> at
> >>
> org.eclipse.jface.viewers.StructuredViewer.handleOpen(StructuredViewer.jav
> a:1079)
> >>
> >> at
> >>
> org.eclipse.jface.viewers.StructuredViewer$6.handleOpen(StructuredViewer.j
> ava:1183)
> >>
> >> at
> >>
> org.eclipse.jface.util.OpenStrategy.fireOpenEvent(OpenStrategy.java:263)
> >> at
> >> org.eclipse.jface.util.OpenStrategy.access$2(OpenStrategy.java:257)
> >> at
> >>
> org.eclipse.jface.util.OpenStrategy$1.handleEvent(OpenStrategy.java:297)
> >> at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
> >> at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:938)
> >> at
> >> org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3682)
> >> at
> >> org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3293)
> >> at
> >> org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2389)
> >> at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353)
> >> at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2219)
> >> at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
> >> at
> >>
> org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:28
> 9)
> >>
> >> at
> >>
> org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:461
> )
> >>
> >> at
> >> org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
> >> at
> >>
> org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplicatio
> n.java:106)
> >>
> >> at
> >>
> org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.jav
> a:169)
> >>
> >> at
> >>
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplicatio
> n(EclipseAppLauncher.java:106)
> >>
> >> at
> >>
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(Eclipse
> AppLauncher.java:76)
> >>
> >> at
> >>
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:36
> 3)
> >>
> >> at
> >>
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:17
> 6)
> >>
> >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> >> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> >> at java.lang.reflect.Method.invoke(Unknown Source)
> >> at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:508)
> >> at org.eclipse.equinox.launcher.Main.basicRun(Main.java:447)
> >> at org.eclipse.equinox.launcher.Main.run(Main.java:1173)
> >> at org.eclipse.equinox.launcher.Main.main(Main.java:1148)
> >>
> >>
> >>
> >>
> >
More information about the antlr-interest
mailing list