[antlr-interest] plugin, ITokenScanner, nextToken --> SWT BUG
elekis
elekis at gmail.com
Thu Apr 10 00:57:07 PDT 2008
Nev a écrit :
> Are you running on Windows. Likely to have \r\n at end of each line.
> Nev
>
>
Yes I run under windows. but is it in eclipse or in antlr that I have
to patch??
thanks
a++
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
>> bounces at antlr.org] On Behalf Of elekis
>> Sent: Thursday, 10 April 2008 4:07 PM
>> To: Jan van der Ven; ANTLRDev Interest
>> Subject: Re: [antlr-interest] plugin, ITokenScanner, nextToken --> SWT BUG
>>
>> re-hi,
>>
>> I thing I found where is the prob, but I have no idea how to patch it.
>> there is the code I wanna scan:
>>
>> FROM fslkdfjsl IMPORT lskfjslk;
>> FROM fslkdfjl IMPORT ldsfkjdjkfl;
>>
>> easy isnt -it
>>
>> and there is what getNextToken, getoffset and getlength return.
>> TOKEN -->FROM<--
>> OFFSET: 0
>> SIZE: 4
>> --------------------------------
>> TOKEN --> <--
>> OFFSET: 4
>> SIZE: 1
>> --------------------------------
>> TOKEN -->fslkdfjsl<--
>> OFFSET: 5
>> SIZE: 9
>> --------------------------------
>> TOKEN --> <--
>> OFFSET: 14
>> SIZE: 1
>> --------------------------------
>> TOKEN -->IMPORT<--
>> OFFSET: 15
>> SIZE: 6
>> --------------------------------
>> TOKEN --> <--
>> OFFSET: 21
>> SIZE: 1
>> --------------------------------
>> TOKEN -->lskfjslk<--
>> OFFSET: 22
>> SIZE: 8
>> --------------------------------
>> TOKEN -->;<--
>> OFFSET: 30
>> SIZE: 1
>> --------------------------------
>> PROBLEME HERE WHY SIZE = 2
>> TOKEN -->
>> <--
>> SIZE: 2
>>
>> etc....etc...
>>
>> there is my switch and WS is define like that
>> WS :(' '|'\r'|'\t'|'\u000C'|'\n')+;
>>
>> switch (antCurrentToken.getType()) {
>> case LyaflplugLexer.WS:
>> return Token.WHITESPACE;
>> case LyaflplugLexer.TAG_POINT_VIRGULE:
>> return Token.UNDEFINED;
>> case LyaflplugLexer.TAG_END:
>> case LyaflplugLexer.TAG_DEFINITION_MODULE:
>> case LyaflplugLexer.TAG_FROM:
>> case LyaflplugLexer.TAG_IMPORT:
>> return keyword;
>> case LyaflplugLexer.Ident:
>> return ident;
>> default:
>> return Token.UNDEFINED;
>> }
>>
>> What I don't understand is why the size is 2??? there only one \n. any
>> idea??
>> thanks
>>
>> a+++;
>>
>>
>>
>> Jan van der Ven a ?crit :
>>
>>> Dear Julien,
>>>
>>> I remember having the same kind of problems. Please check the way we
>>> fixed it here:
>>> http://quantum.cvs.sourceforge.net/quantum/quantum-
>>>
>> plugin/src/com/quantum/editors/SQL2ScannerWithLexer.java?view=log
>>
>>> If I remember correctly, the setRange is sometimes called with illegal
>>> values for offset and/or length.
>>>
>>> Happy coding,
>>>
>>> Jan
>>>
>>> julien wrote:
>>>
>>>> hi all, I try to make a plugin who can colorize some line like
>>>>
>>>> FROM lkfsjdl IMPORT lskdfjl;
>>>> FROM lkfsjdl IMPORT lskdfjl;
>>>>
>>>> Based on my own grammar created with ANTLR 3.0.
>>>>
>>>> for that, I overrided ITokenScanner.
>>>>
>>>> public IToken nextToken() {
>>>>
>>>> antCurrentToken = lexer.nextToken();
>>>> if (antCurrentToken.getType() == org.antlr.runtime.Token.EOF)
>>>> return Token.EOF;
>>>> switch (antCurrentToken.getType()) {
>>>> case LyaflplugLexer.TAG_END:
>>>> case LyaflplugLexer.TAG_DEFINITION_MODULE:
>>>> case LyaflplugLexer.TAG_FROM:
>>>> case LyaflplugLexer.TAG_IMPORT:
>>>> return keyword;
>>>> default:
>>>> return ident;
>>>> }
>>>> }
>>>>
>>>> The problem is : if the file is empty then I can create twho line
>>>>
>>>> FROM lkfsjdl IMPORT lskdfjl;
>>>> FROM lkfsjdl IMPORT lskdfjl;
>>>>
>>>> he color eatch word etc... everything is perfect but if I save and I
>>>> reopen the file he bug cause theres are two line who are the same.
>>>> (if there is only one, no prob).
>>>>
>>>> I ve tried to debug and he call every token. but after that there is
>>>> a exception error Argument No valid.
>>>>
>>>> is any body has a idea: I there is the class code
>>>> http://paste.ubuntu-nl.org/62601/
>>>>
>>>> thanks
>>>>
>>>> a++
>>>>
>>>> PS: this message is sended on antlr-interest and eclipse.platform
>>>> -----------------------------------------------------------------------
>>>>
>>>> PPS: there is the stack.
>>>>
>>>>
>>>> java.lang.IllegalArgumentException: Argument not valid
>>>> at org.eclipse.swt.SWT.error(SWT.java:3547)
>>>> at org.eclipse.swt.SWT.error(SWT.java:3481)
>>>> at org.eclipse.swt.SWT.error(SWT.java:3452)
>>>> at
>>>> org.eclipse.swt.custom.StyledText.setStyleRanges(StyledText.java:7776)
>>>> at
>>>> org.eclipse.swt.custom.StyledText.setStyleRanges(StyledText.java:7870)
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.TextViewer.applyTextPresentation(TextViewer.java:44
>> 29)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.TextViewer.changeTextPresentation(TextViewer.java:4
>> 481)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.presentation.PresentationReconciler.applyTextRegion
>> Collection(PresentationReconciler.java:580)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.presentation.PresentationReconciler.processDamage(P
>> resentationReconciler.java:569)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.presentation.PresentationReconciler.access$3(Presen
>> tationReconciler.java:565)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.presentation.PresentationReconciler$InternalListene
>> r.inputDocumentChanged(PresentationReconciler.java:119)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.TextViewer.fireInputDocumentChanged(TextViewer.java
>> :2685)
>>
>>>> at
>>>> org.eclipse.jface.text.TextViewer.setDocument(TextViewer.java:2734)
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:5
>> 18)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.text.source.SourceViewer.setDocument(SourceViewer.java:4
>> 73)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.texteditor.AbstractTextEditor.initializeSourceViewer(Abstra
>> ctTextEditor.java:3786)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.texteditor.AbstractTextEditor.createPartControl(AbstractTex
>> tEditor.java:3332)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.texteditor.StatusTextEditor.createPartControl(StatusTextEdi
>> tor.java:53)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.texteditor.AbstractDecoratedTextEditor.createPartControl(Ab
>> stractDecoratedTextEditor.java:378)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.j
>> ava:661)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:42
>> 6)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartRefere
>> nce.java:592)
>>
>>>> at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:299)
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.presentations.PresentablePart.setVisible(Presentab
>> lePart.java:179)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(Pr
>> esentablePartFolder.java:268)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(Left
>> ToRightTabOrder.java:65)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectP
>> art(TabbedStackPresentation.java:400)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.j
>> ava:1256)
>>
>>>> at
>>>> org.eclipse.ui.internal.PartStack.setSelection(PartStack.java:1209)
>>>> at org.eclipse.ui.internal.PartStack.showPart(PartStack.java:1604)
>>>> at org.eclipse.ui.internal.PartStack.add(PartStack.java:499)
>>>> at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:103)
>>>> at org.eclipse.ui.internal.PartStack.add(PartStack.java:485)
>>>> at org.eclipse.ui.internal.EditorStack.add(EditorStack.java:112)
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorSashContainer.addEditor(EditorSashContainer.
>> java:63)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorAreaHelper.addToLayout(EditorAreaHelper.java
>> :217)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorAreaHelper.addEditor(EditorAreaHelper.java:2
>> 07)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorManager.createEditorTab(EditorManager.java:7
>> 74)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorManager.openEditorFromDescriptor(EditorManag
>> er.java:673)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.EditorManager.openEditor(EditorManager.java:634)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.busyOpenEditorBatched(WorkbenchPage.
>> java:2737)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.busyOpenEditor(WorkbenchPage.java:26
>> 51)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.access$13(WorkbenchPage.java:2643)
>>
>>>> at
>>>> org.eclipse.ui.internal.WorkbenchPage$10.run(WorkbenchPage.java:2595)
>>>> at
>>>> org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:67)
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2590)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2574)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.WorkbenchPage.openEditor(WorkbenchPage.java:2565)
>>
>>>> at org.eclipse.ui.ide.IDE.openEditor(IDE.java:644)
>>>> at org.eclipse.ui.ide.IDE.openEditor(IDE.java:603)
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUt
>> ility.java:286)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.internal.ui.javaeditor.EditorUtility.openInEditor(EditorUt
>> ility.java:139)
>>
>>>> at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:194)
>>>> at org.eclipse.jdt.ui.actions.OpenAction.run(OpenAction.java:175)
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.ui.actions.SelectionDispatchAction.dispatchRun(SelectionDi
>> spatchAction.java:268)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.ui.actions.SelectionDispatchAction.run(SelectionDispatchAc
>> tion.java:244)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.internal.ui.packageview.PackageExplorerActionGroup.handleO
>> pen(PackageExplorerActionGroup.java:316)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jdt.internal.ui.packageview.PackageExplorerPart$5.open(Package
>> ExplorerPart.java:613)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.viewers.StructuredViewer$2.run(StructuredViewer.java:820
>> )
>>
>>>> at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:37)
>>>> at org.eclipse.core.runtime.Platform.run(Platform.java:857)
>>>> at org.eclipse.ui.internal.JFaceUtil$1.run(JFaceUtil.java:46)
>>>> at org.eclipse.jface.util.SafeRunnable.run(SafeRunnable.java:199)
>>>> at
>>>>
>>>>
>> org.eclipse.jface.viewers.StructuredViewer.fireOpen(StructuredViewer.java:
>> 818)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.viewers.StructuredViewer.handleOpen(StructuredViewer.jav
>> a:1079)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.viewers.StructuredViewer$6.handleOpen(StructuredViewer.j
>> ava:1183)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.jface.util.OpenStrategy.fireOpenEvent(OpenStrategy.java:263)
>>
>>>> at
>>>> org.eclipse.jface.util.OpenStrategy.access$2(OpenStrategy.java:257)
>>>> at
>>>>
>>>>
>> org.eclipse.jface.util.OpenStrategy$1.handleEvent(OpenStrategy.java:297)
>>
>>>> at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:66)
>>>> at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:938)
>>>> at
>>>> org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3682)
>>>> at
>>>> org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3293)
>>>> at
>>>> org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2389)
>>>> at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353)
>>>> at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2219)
>>>> at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
>>>> at
>>>>
>>>>
>> org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:28
>> 9)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:461
>> )
>>
>>>> at
>>>> org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
>>>> at
>>>>
>>>>
>> org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplicatio
>> n.java:106)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.jav
>> a:169)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplicatio
>> n(EclipseAppLauncher.java:106)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(Eclipse
>> AppLauncher.java:76)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:36
>> 3)
>>
>>>> at
>>>>
>>>>
>> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:17
>> 6)
>>
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>>>> at java.lang.reflect.Method.invoke(Unknown Source)
>>>> at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:508)
>>>> at org.eclipse.equinox.launcher.Main.basicRun(Main.java:447)
>>>> at org.eclipse.equinox.launcher.Main.run(Main.java:1173)
>>>> at org.eclipse.equinox.launcher.Main.main(Main.java:1148)
>>>>
>>>>
>>>>
>>>>
>>>>
>
>
>
More information about the antlr-interest
mailing list